NoSQL Zone is brought to you in partnership with:

Eric is an artist and storyteller who boasts his first computing experience juggling drivers into memory to run as many DOS games as possible without rebooting. Since then he has become a creative, patient reverse engineer who operates on the basic premise that something never has to be the way that it is. Eric has a B.S. in Computer Science from the Georgia Institute of Technology and his experience in the ETL layer across a broad range of industries has instilled in him an appreciation for semantics and the belief that all of mankind's problems are related to our adherence to inaccurate representations of reality. Eric is a DZone MVB and is not an employee of DZone and has posted 8 posts at DZone. You can read more from them at their website. View Full User Profile

MongoDB Performance Tuning with Dex

09.14.2012
| 12849 views |
  • submit to reddit

(also posted to the 10gen blog: here)

Greetings Adventurers!

I’m excited to report that Dex (github) is now equipped with his first planned upgrade. For those of you who haven’t met him, Dex is an open-source python tool that suggests indexes for your MongoDB database. The initial release of Dex supported logfile-based analysis only. Now, Dex can be run remotely by leveraging MongoDB’s built-in database profiler. This new feature is invoked with the -p or --profile option. When you run Dex with -p, it connects to the specified MongoDB database and analyzes the queries already logged to your system.profile collection.

As the diagram below shows, this is particularly good news for MongoLab’s shared plan customers and anyone who does not have direct terminal access to their database machine.

In case you missed my introduction of Dex at the San Francisco MongoDB User Group Meetup, I’ll be presenting Dex at the Silicon Valley MongoDB Users Group on July 17 at 10gen’s offices in Palo Alto!

Here’s a quick set of steps to get you started.

  1. If you haven’t already, get Dex:
    sudo pip install dex

    Or, if you already have Dex:

    sudo pip install dex --upgrade
  2. Log into your database through the mongo shell and run db.setProfilingLevel(1). If your MongoDB is hosted with us at MongoLab, you can also enable profiling through our UI.
  3. Let your app run for a while. With profiling enabled, MongoDB deposits documents into system.profile. By default, each of these documents represents a database operation that took more than 100ms to complete. For apps that perform specific operation at specific times, you will need to profile during those times. Once you feel that your profile collection is populated with a representative set of data, you’re ready to run Dex!
  4. Run Dex with --profile or -p (instead of -f):
    dex -p <mongodb-uri>

    Where <mongodb-uri> is your database connection string (ex: mongodb://me:mypassword@myserver.mongolab.com:27227/mydb)

  5. Dex outputs index recommendations and corresponding creation syntax.  Because Dex relies on heuristics that don’t take your data into account, you’ll want to validate and sanity check Dex’s suggestions before implementing.
  6. Run db.setProfilingLevel(0) to disable profiling when you’re done. Profiling requires a small bit of overhead and is entirely diagnostic, so you don’t need to leave it running. If you like, you can also drop the system.profile collection afterwards.
  7. Enjoy!

As always: if you have any questions, bug reports, or feature requests, please email us at support@mongolab.com.

Until next time, good luck out there!

Sincerely,
Eric@MongoLab

Published at DZone with permission of Eric Sedor, author and DZone MVB. (source)

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)