Peeking Behind the Neo4j Lucene Index

Did you know you can write Javascript in the Neo4j console to access the Neo4j API?
Try it. Open up your Neo4j Web Admin Console and type:

neo4j-sh (0)$ eval db
EmbeddedGraphDatabase [data/graph.db]

OMG! I know, Neo4j is crazy. So much to play with, I’ve been at it for a few years and I haven’t even dug into this area. What else can we do here?

neo4j-sh (0)$ eval db.config
{ephemeral=false, keep_logical_logs=true, neo4j.ext.udc.source=server, neostore.nodestore.db.mapped_memory=368M, neostore.propertystore.db.arrays.mapped_memory=1277M, neostore.propertystore.db.mapped_memory=1326M, neostore.propertystore.db.strings.mapped_memory=1127M, neostore.relationshipstore.db.mapped_memory=1624M, node_auto_indexing=true, node_keys_indexable=name,title,released,tagline, remote_shell_enabled=true, store_dir=data/graph.db}

Pretty cool right? I have a Matrix movie graph loaded, let’s try to pull up a node by index:

eval db.index().forNodes("node_auto_index").get("title","The Matrix").getSingle().getProperty("title").toString();
The Matrix

You can go wild here, but what I really wanted to do is take a peek behind the API curtain and see what’s really going on with the Neo4j Indexes.

Luke is a handy development and diagnostic tool, which accesses already existing Lucene indexes and allows you to display and modify their content in several ways:

• browse by document number, or by term
• view documents / copy to clipboard
• retrieve a ranked list of most frequent terms
• execute a search, and browse the results
• analyze search results
• selectively delete documents from the index
• reconstruct the original document fields, edit them and re-insert to the index
• optimize indexes
• and much more…

You can download the Luke 4.0.0-ALPHA standalone binary and run it with:

java -jar lukeall-4.0.0-ALPHA.jar 

Then point it to one of your Neo4j Indexes (here I am using the node_auto_index since I have configured my Neo4j instance to use it).

/neo4j/data/graph.db/index/lucene/node/node_auto_index/segments.gen

If your database is running, you will want to open it in Read-Only mode.

Here you can see the fields that are indexed as well as some statistics about them:

You browse through each record in the index and can even query the syntax directly and see what’s inside:

Take a look at this blog post for a more detailed look at Luke’s capabilities.

Be careful what you do here, and I think it’s better to play with a copy of your database instead of making any changes this way to the live database.