Over the last week, we’ve shipped a couple key Android App, database, and server updates to SimpleCrew.
The updates addressed a couple of the most urgent scaling issues we’ve faced to date.
Here’s a look into the recent updates, and the problems they addressed.
System-Wide Server Crashing – Android App and Server Update.
In our update a few weeks back, I disclosed a heavy load issue on our server that was causing system-wide crashing.
During this time, some users were experiencing login issues, photo upload failures, and slow connections.
Over the past few weeks, Mike identified the issue as a flaw in an old version of our Android app where, when a photo upload failed due to low service coverage, the app would get into a never ending loop of re-posting the photo to our server.
At the time, since the server was forced to accept any photo being uploaded, the never-ending upload loop would cause the SimpleCrew server to crash and be inaccessible to other users.
Mike made a number of server updates to combat the self-destructive mobile app, and using insights from our analytics/tracking system Mixpanel, we were able to identify which user’s Android apps were stuck in infinite loops, and contacted them directly to ask them to re-boot their apps.
Finally, we contracted a mobile app developer to update our Android app and resolve the issue completely. The updated Android app was released this weekend. Download it in the Android Market.
1000+ Photos Issue – Database Update.
This was an interesting issue that some of our biggest customers were expeirencing when their campaigns would reach 1000+ photos.
Our original database design (MongoDB for the techies) was not optimized for querying large numbers of photos.
The design was optimized for more flexibility, but as we came to find, it would eventually break down when campaigns reached 1000+ photos.
Over the last few weeks, Mike was able diagnose the issue, and built updates to the database schema design and application.
Then, this weekend – the trickiest part – Mike was able to migrate all the existing campaigns to work with the new system.
These scaling issues presented some of the most urgent challenges we’ve faced so far in our (brief) history, so many thanks to those of you who’ve been patient with us and helped us with feedback as we worked through them.
Today, you guys are submitting over 1,000 photos on our system every day, and your campaigns are reaching 1000+ photos with greater and greater frequency.
Though the issues were frustrating, we reminded ourselves that in the grand scheme of things, scaling issues are an indcation of a useful product being used.
For that, we’re thankful, and will continue to work hard to make SimpleCrew the best solution for your business.
– Alan VanToai