coding, mongodb, nodejs, Uncategorized

Recently open sourced project – toronto open data

The motivation behind this project is

 

  • Building a very capable api with geo lookups
  • Toronto Open Data api -specifically city services is very slow
  • User interface as a proof of concept which exploits the use of new API
  • Open sourced for others to use and improve

Technology stack:

 

  • MongoDB
  • NodeJS, rest api, mongoose
  • AngularJS (bootstrap, routes)

 

GiHub link

Share This:

architecture, bigdata, mongodb

MongoDB 3.0 worth the upgrade

MongoDB 3.0 (dev version of 2.8) has been almost a year in the making and there are several prominent changes that are worth mentioning:

 

  1. Document level locking – changes to those large json documents (aka rows ) will work similarly to regular RDBMS. Previously you had to either queue or create a custom way of handling inconsistencies.
  2. Document has a hard limit of 16megs (reasonable) and that allows your app to create massive amount of data that will require sharding your servers. A new storage engine WiredTiger will provide better compression for storage, allegedly up to 80% , shed those Gigs of data Foursquare!
  3. Besides other significant improvements, a new tool is introduced named Ops Manager, uncertain if it would be available to all or MMS subscribers.
  4. Looking into MongoDB Jira , there is a large set of bugs that have been fixed, better error handling in GEO lookups, syncing issues between nodes and others.

 

 

A few notes:

 

In order to take advantage of a new pluggable storage engine wiredTiger you need to explicitly configure it as such:

 

mongod --storageEngine wiredTiger

Overall great job!

 

 

Share This:

architecture, mongodb

Why i still love MongoDB

After reading a Slashdot article on MongoDB vs DynamoDB (chosen) i’d like to summarize why DynamoDB became less attractive to me:

 

  • Architect/Developer chose to serialize a collection of objects into JSON and save that field as one column – this screams future problems in searching and simple lookups
  • Apparently (this may have changed) there is 64Kb limit on the item , some people start to compress data (json)
  • Manually rebuilding indexes – no comment here
  • Pricing model has to be predefined – number of reads per second, although affordable vs dedicated cloud based mongodb deployment (given that i am not sure if performance numbers are the same)
  • No way to embed objects , you’d have to create infinite amount of columns (similar to cassandra)

 

 

Share This:

architecture, coding, javascript, mongodb

Skedul.In project is wonderful – power of google API and ruby

Not always do you get to work with wonderful client that know what they want, but i have. Skedul.In soon to be launched in production mode (currently in beta) is a simple yet great idea: have single place to create all your events at Once , select your google calendar, use your google contacts and you are done, all events are created, invitation sent, your calendar updated.

 

Techie stuff:

Ruby 2.0

Rails 3.2.X

MongoDB – Can’t live without it , also used for session storage

Completely relies on OpenAuth , you have to login with your Google Account

Google Calendar API, Contacts API

Hosted on Heroku

 

Couple of screenshots :

Screenshots

Share This:

architecture, mongodb

MongoDB in 2012

My tale of MongoDB

I’ve had a great time with MongoDB in 2012. I’ve replaced several mysql DBs with nicely organized mongoDB collections.
I’ve developed an app using NodeJS (see mongoose) and stored massive data in mongodb.

My favourite host: MongoLab.com , 500megs free per db and great engineering minds work there.

I’ve built and deployed RubyOn Rails app with storage on mongodb and it was fantastic, Mongoid is the gem i used for models. Deployed on Amazon AWS elasticBeanstalk proved immensely fantastic , after tinkering with python tool.

I have built a mobile app for iphone/ipad powered by json data , from MongoDB and managed by a simple rails app.

10gen is awesome

Release of MongoDB 2.2 was big! Performance and features of the release deserve almost a major release.

The folks at 10gen are also very friendly, i received my promotional material (stickers, book and discount coupons for books) that i happily distribute as i spread the word.

Share This: