Paths, Entities, and Types in Spring Data for Neo4j (also, I'm back)

I know it's been awhile since my last post, but, I assure you--one and all--that I have not left you kind folks for good.  I can also assure you that the cause of my literary absence has everything to do with being loaded down with work and is not a result of my gallivanting around the world whilst trying strange and wonderful new beers.

(Boy, do I ever wish that was the case.)

Strange AND wonderful!
[Courtesy: FOX]

No, I've managed to find some time in which to write a bit about something I've been looking into concerning SDN: Paths with multiple entity types.

So, without further ado, let's begin.

Yes, quite.
[Courtesy: samuelrunge.com and Monty Python)

Disclaimer

While the topics I'm about to go into could very well have solutions to them that I have yet to uncover, I'm presenting my findings and questions in the hopes that not only will someone find the discussion interesting/useful, but, that it might also help lead me to better solutions.

So, please do read on and try to keep the caveat above in mind.

Retrieving Heterogeneous Paths

If you'll recall from several posts ago, I had been attempting to write a web application based around the concept of a simple recommendation engine and a software retailer (a la Babbage's from the last century).  The implementation was being done using a Spring-based Java stack with Neo4j as the data store.

I had gotten to a point where I was able to load data into Neo4j via a method not unlike the one from Cineasts' (which can be found as part of the Spring Data Neo4j Reference).  A quick recap of the domain model follows below.

The domain model in question.

One important task of any recommendation engine is the ability to suggest entities that are relevant to a given starting point via some sequence of relationships.  Implementing such an engine, while quite doable, is something that I will eventually get to.

Another, more simple, task is to be able to see how two entities are related, i.e. "how do I get from point A to point B?".  As an example, one such question might be, "How is game A related to game B, even though the difference in publication dates is large?"  And yes, this type of question is extremely similar to the "Six Degrees of Separation" question.

I bet he's a gamer.
[Couresty: Wikipedia]

Some Basic Code

You might also recall that I had implemented a GameRepository class with the following signature:

 
public interface GameRepository extends GraphRepository<Game>, RelationshipOperationsRepository<Game>

With the above repository extending the GraphRepository and RelationshipOperationsRepository interfaces, we are provided with a host of cool (and handy) methods out of the box (tip: make sure you're comfortable with the "convention over configuration" paradigm, as there is a bit of magic that goes on with those OOTB methods).

As you'd expect, we can also add additional methods to the interface.  One example of a method you might want to add could be a custom Cypher query that returns all Game nodes with a particular property (the implementation of this is outside the scope of this post, but, it's actually pretty simple; if people want to see it, just shoot me a note!).

However, today we're looking to address the "Six Degrees of Separation" question (minus the limit on the degrees of separation), i.e. "how are node A and node B related"?

So let's give this a (very simple) shot:

 
@Query("START n=node(1), x=node(97) MATCH p = shortestPath( n-[*]-x ) RETURN p")
Iterable<EntityPath<Game, Game> > getPath();

Given that the nodes with IDs 1 and 97 are "Game" nodes, the Cypher query above is essentially determining how the two nodes are related.

(For the sake of this post, I'm ignoring the fact that there could be multiple "shortest paths" between the two nodes as it has little bearing on the goal of this post.)

Quickly going over the return type, SDN allows us to return back an EntityPath given a starting node type and an ending node type which, in this case, is the Game type.  An EntityPath is capable of storing nodes and their relationships as part of a Cypher path.  The Iterable portion of the return type is necessary unless you want to use EndResult instead of Iterable.

We can then access the individual path(s) via the Iterable return type.

(NOTE: There is currently a bug with SDN that throws an exception when calling a single path's .nodes() or .nodeEntities().  This bug has been around since SDN 2.1.RC4.)

Traversing the Returned Path

There's a reason my explanation of the code used stops where it does.  Those of you with a keen eye and/or are familiar with OOP/OOD will identify a potentially big stumbling block: How do you iterate through a path of nodes and/or relationships with potentially wholly disparate, unrelated types?  Given that this is SDN and is geared towards integrating easily with Java and POJOs, the issue becomes apparent.

How do we solve this?

Solution 1

Make sure the nodes you are after all either implement a common interface or are derived from a common class.

Seem too good to be true?  You're right, because it is.  While there may be some scenarios in which this solution might work, it is often the case in a graph database the nodes/relationships are entirely unrelated concepts/types, e.g. Game and Customer, or, Game and Developer.  This separation would imply that there are likely methods and attributes that are specific to a given type that would have no business being in a shared superclass or interface.

Solution 2

Employ some form of reflection.

There it is: The "r" word.  Reflection is generally quite costly and so immediately handicaps its appeal.

A "poor man's" reflection might be to implement a series of "if/else" blocks to check types and perform some appropriate casting.  I think we can see that this could and would become very ugly and difficult to maintain.

What about full-on automated reflection?  Well, we run into a bit of a snag with that, too: In a typical assignment operation, we have a left-hand side (LHS) and a right-hand side (RHS), e.g. TheClass theClassInstance = new TheClass();.  The RHS of the assignment is fairly straight forward with reflection.  Since SDN persists a __type__ attribute/property into a given node/relationship, we can fetch it and use it for casting (since it's typically a fully-qualified type name).  It might look something like this:

// n = Node object in question
String theType = (String)n.getProperty("__type__", "");

// we could then make the RHS of the assignment something like: (LHS) = template.convert(n, class<theType>;

But what about the LHS?  Without an "if/else" block, how can we treat the returned string theType as a first-class citizen that would declare the type for the LHS?  As far as I'm aware, there is no way to do this (of course there might be ways I've just not seen, but, I have a feeling they'd be just as expensive as the rest of the assignment).  Java is a strongly-typed language, and so I'm sure most of us would expect this outcome.

So we see that this "solution" isn't really much of one.

Solution 3

Somehow modify the query to work with a @MapResult-annotated interface to deal with the results (note: At least as of SDN 2.3.1, @MapResult has been deprecated in favour of @QueryResult.  I haven't done too much with @QueryResult so your mileage may vary).  This obviously requires more knowledge ahead of time of the types of paths, nodes and relationships you're planning on returning which may limit the kinds of heterogeneous queries you can execute.

So What Else Can Be Done?

I recently attended GraphConnect 2013 in New York City where I had a chance to meet up with Michael Hunger (a name which should require little or no introduction in the graph database/Spring Data continuum).  We had a great conversation about the very subject of this post.

His overall insights into Spring Data and its purpose, merits and detractors were very helpful, especially from a conceptual standpoint.

The number one point to take away--and perhaps it's quite obvious but it's worth reiterating--is this: Spring Data is not a magic bullet.  Given the differences in concepts here (i.e. a strongly-typed, object-oriented language and a graph-based, schema-free data store), there is bound to be limitations.

Spring Data's strong point is ease of integration.  A typical use case for SDN is likely to be one where relatively few nodes/relationships are needed to be returned.  SDN is not meant to necessarily "explore" graphs.

In order to truly resolve such differences, it would seem to me to make more sense to either layer SDN on top of another layer of abstraction, or even to go direct to the Neo4j API.

Perhaps an even better approach would be to use a Domain-Specific Language (DSL) such as Groovy or JRuby; something that is much more loosely-typed, flexible, and still able to be integrated into the Java stack.

(Shameless plug: Check out Pacer, a powerful, JRuby-based graph traversal engine.)

Summary/Conclusion

In this post, we have seen that exploring subgraphs and paths with SDN is not as straightforward as we'd perhaps like; however, it is clear that SDN was not built to accomplish such features (at least not yet).

Spring Data for Neo4j's strong suit is ease of integration.  As should be evident from this post and others, it is easy and straightforward to get SDN into existing/legacy Java applications, and to quickly stand-up Java-based applications that rely more on "end results" than "exploration" per se.

Ok, folks; as always, I definitely welcome comments,feedback, and questions.  If you can think of a better way to approach this kind of problem space or even if I have something wrong here, please do let me know and I'll be sure to make good use of such feedback.

Thanks for reading, and we'll see you on the next post!

Spring Data Neo4j 2.3.1.RELEASE and Neo4j 1.9.4 Upgrading

I'm back with a quick post (with more to come soon).

I was in the middle of upgrading my little test project to a newer version of Spring Data Neo4j and Neo4j itself when I came across a few little points that others might find useful (though it should be noted that Neo4j is set to release 2.0 very soon and is currently doing milestone releases).

I upgraded SDN to 2.3.1.RELEASE and Neo4j (all aspects of it, including Cypher) to 1.9.4.

Here are a couple "gotchas" I encountered:

Dependencies

It would seem that CGLIB has been moved out of one of the Neo4j or SDN dependencies; however, I also found that--with the SDN/Neo4j combination I'm using--that a specific version is required, namely 2.2.2.

Adding this bit into my POM fixed things up nicely:

<dependency>

    <groupId>cglib</groupId>

    <artifactId>cglib</artifactId>

    <version>2.2.2</version>

</dependency>

No bean named 'graphDatabaseService' is defined

This one was fun.  As confirmed in this Neo4j forum thread (which actually uses Neo4j 1.7 and SDN 2.1.0.Build-Snapshot), when configuring Neo4j in an application context, the bean ID for the graph database service (whether it's embedded or from a server) must be "graphDatabaseService", similar to this:

<bean id="graphDatabaseService"

class="org.springframework.data.neo4j.rest.SpringRestGraphDatabase">

<constructor-arg index="0" value="http://someserver:7474/db/data" />

</bean>

If this little nuance is overlooked, you could very well see exceptions when, say, starting up your application server with your SDN-based application.

In my case, Maven compiled my WAR file just fine, but, starting up Tomcat produced a slew of exceptions.

It would seem that this is an ongoing issue, though, perhaps it's a necessary change from the SDN folks.  We'll just have to see!

Hopefully some people find these tidbits useful.

We'll see you on the next post!

DZone Love

First off, I promise I'm still alive.  Work has just been absolutely insane lately and it doesn't allow me a lot of time to get to blog about the graph stuff I love so much.  I'm hoping that changes soon!

To that end, it would seem that the good folks at DZone have sent me a goodie box (or, as I call it, a "swag bag" because it rhymes and I'm easily amused) in order to give me a kick in the rear end.  Or to say, "thank you."  Either way, it's pretty awesome.

Here's a picture I took of me this morning (pre-coffee/anything) with the shirt they sent me:

I was a bit confused at first why they would send a "vɘb" shirt.  When a colleague of mine told me that it's ironic that I was wearing a "dev" shirt since I haven't been a plain ol' dev in awhile, I snorted and told him, "Shows what you know!  It's a vɘb shirt!"  He then shook his head and walked away.

It wasn't until I was getting some water that I put two and two together.

(That story may or may not have actually happened.  Either way, the shirt is awesome and so is the other stuff that came with it!)

Ok, that's it for this short post.  I wanted to express my thanks to the people over at DZone.  Thanks for having me as part of your MVB program!

I'll see you all next time!

Spring Data Neo4j, @MapResult, Cypher, Casing and You!

A quick tip for those of you who are using Cypher with Spring Data Neo4j (SDN):

If you're using the @MapResult way in your Neo4j repositories, be careful of what you use in the corresponding @ResultColumn annotations.

For example, let's say you have the following repository definition (assume that you don't want to use the built-in findAll() method in this case; this example can be extended to other mapping results WLOG):

 
public interface MyRepository extends GraphRepository, RelationshipOperationsRepository {
 @Query("START n=node:__types__(className='com.yourorg.yourproject.entities.MyModel') RETURN COLLECT(n)")
 MyModelData getAllMyModels();
 
 @MapResult
 public interface MyModelData
 {
  @ResultColumn("COLLECT(n)")
  Iterable getMyModels(); 
 }
}

What you'd expect getAllMyModels() to do is to simply return all of those nodes that meet the Cypher criteria (note that the Cypher query above is very similar to what is generated by SDN in its findAll() method; the index referenced is, in fact, created and persisted by SDN).

However, if you call this code, you will get an error similar to the following:

"org.springframework.data.neo4j.support.conversion.NoSuchColumnFoundException: Expexted a column named COLLECT(n) to be in the result set."

You're probably scratching your head and asking yourself at this point, "But I am returning 'COLLECT(n)'!  It's right there in the Cypher query!"

And you're completely right--it is there!

However, try running that same query in the Neo4j web console (wherever your graph database is residing).

Dig through the results, and you'll see that the column returned isn't, in fact, "COLLECT(n)", but, "collect(n)".

Yep, you got it!  It's case sensitive!

So if you change

@ResultColumn("COLLECT(n)")

...to...

@ResultColumn("collect(n)")

...(note that casing of "collect") you'll be good as gold.

Remember that the next time you're diving deep into SDN.

And, as an update, I continue to work on the project I started back in June, and am finally making some headway.  I'm coming across some interesting stuff, and I hope to share more in the coming weeks.

We'll see you on the next post!

Persisting to Neo4j via Spring Data (or, "Aren't We Persistent?")

Hi gang!

Ok, I'm back with a new post, this time with a post about a couple quirks I ran into while implementing some test cases for Spring Data using Neo4j.  They're not bugs by any stretch; it's just new behaviour to get used to as you venture into the Spring Data world (which I'm loving, by the way).

During my copious amounts of downtime (that should be read while imagining me rolling my eyes so hard that I fall over backwards in my chair), I've been putting together a little playground for me to mess around with and play with Spring Data.

It's definitely evolving and changing as I change things up and try out new ideas, and I fully plan on sharing more about this in future posts.

For now, though, I'm just discussing a couple potential pitfalls newcomers to Spring Data might fall prey to (please pardon the prepositional phrase; I'm sure it won't be the last one).

Background
I've wanted to play with Spring Data a bit more seriously for some time now, so I started a few weeks ago and, I have to say, I'm loving every second of it.  (I'm already a huge Spring fan, and the annotations continue to make my life easier.)

My sandbox goes something like this: After having gone through the docs for Spring Data (especially "Good Relationships"), I thought I'd try out something similar for myself, borrowing the whole "store" concept (as it seems to me to be the best, first choice for implementing a graph database).

Instead of using the whole "movie store" concept, I switched to something a little different to avoid total code reuse (I do borrow some code from the link above but modify it an awful lot).

For any geeks around my age (or older), you will remember a certain computer software retailer called Babbage's.  I have fond memories of begging my parents to go into the store every time we passed one, which wasn't often (at least I don't think it was...).  GameStop Corporation went on to purchase Babbage's (and EB Games, and a bunch of other software retailers), so you're unlikely to see a Babbage's by that name.

With that short trip down memory lane finished (more like memory cul de sac), I decided to model my domain after the concept of a software retailer.  My store will cleverly enough be called Von Neumann's (any CS major and most geeks out there are currently groaning at that joke).

Domain Model
Currently, the domain model consists of the following:

Even looking at the UML diagram above, we can see that it's based on a graph model (can you pick out the entities and/or the relationship(s)?).

Test Cases' Setup
After setting up the relevant project (which I did as a Maven project), I set forth creating some tasks using JUnit.  Before creating the actual test cases, I needed to make sure I had my testing context setup.  I also needed away to ensure that any data being persisted was wiped clean after each run.

Fortunately, instead of having to create such functionality for my project, I learned that Neo4j already has a handy solution!  The ImpermanentGraphDatabase.  This little gem can be found in the Neo4j kernel.  Specifically, I added these lines to my POM (you can see the specific version I'm using, too):


<dependency>
<groupId>org.neo4j</groupId>
<artifactId>neo4j-kernel</artifactId>
<version>1.8.M03</version>
</dependency>

...and then adding the following line to my testing context:

<bean id="graphDBService" class="org.neo4j.test.ImpermanentGraphDatabase" destroy-method="shutdown"/>

And presto!  A suitable testing graph database for my test cases!

(Warning: I am using the latest version that I found worked best for me and is compatible with all my other dependencies.  ImpermanentGraphDatabase as available in earlier versions of Neo4j, as well.)

It should also be noted that I make use of both the Neo4j repositories interfaces AND the Neo4jOperations class for persisting and retrieval.

Another note is that I've made the entire test class @Transactional.

Test Cases Proper
I'll list two of them below and a couple of the quirks I noticed.

Ensuring a Customer Can Make a Purchase
This test case consists of creating a Customer object, a couple Game objects, and making sure that Purchases can be created, persisted and retrieved (along with the associated entities). 

 
 @Test
 public void customerCanMakePurchases()
 {
  // setup our game constants
  final int QTY = 1;
  final String GAME_TITLE = "Space Weasel 3.5";
  final String GAME_TITLE_2 = "The Space Testing Game";
  final String GAME_DESC = "Rodent fun in space!";
  final String GAME_DESC_2 = "Tests in space!";
  final int STOCK_QTY = 10;
  final float PRICE = 59.99f;
  
  // setup our customer constants
  final String FIRST_NAME = "Edgar";
  final String LAST_NAME = "Neubauer";
  
  // create our customer for this test
  Customer customer1 = new Customer();
  
  // set the customer's properties (NOTE: "firstName" is an indexed property in the Customer entity, but "lastName" is not!)
  customer1.setFirstName(FIRST_NAME);
  customer1.setLastName(LAST_NAME);

  // create our games for this test
  Stock game1 = new Game(GAME_TITLE, GAME_DESC, STOCK_QTY, PRICE);
  Stock game2 = new Game(GAME_TITLE_2, GAME_DESC_2, STOCK_QTY + 5, PRICE + 5);

First, do the setup.  (And, for the sake of brevity, I'm leaving out the annotated entities.)

Nothing strange going on here--just creating two games and a single customer.  It is worth noting (for later on) that "firstName" is an indexed property of the Customer entity/node.  This means that it is searchable (also recall that Neo4j's default indexing engine is Lucene).

It had to be done.

The games we've chosen are clearly AAA-title games.  These tests should be interesting.

  // save entities BEFORE saving the relationships!
  template.save(game1);
  template.save(game2);
  template.save(customer1);

  // make those purchases! Support our test economy!
  // (NOTE: "makePurchase" actually uses the "template" parameter to persist the relationship, so no need to do it again)
  Purchase p1 = customer1.makePurchase(template, game1, QTY);  
  Purchase p2 = customer1.makePurchase(template, game2, QTY);

Above, we make sure to persist the 2 games and single customer.  We also do this prior to persisting any relationships.  This is necessary.  In this case, I make use of an instance variable called "template" which is actually an instance of  Neo4jOperations.  This is one way of accessing the necessary persistence/retrieval functionality we need.

We then create 2 Purchase objects/relationships (Purchase is actually a relationship entity).  It is also worth noting that, instead of using the "template" object to persist the relationships, I've followed the Neo4j tutorial book "Good Relationships" and attempted another way of doing persistence, i.e. by passing the "template" object into the necessary method and having the method (in this case makePurchase) actually do the persisting of the newly-created Purchase.

Again, both "game1" and "game2" need to be persisted prior to persisting any relationships between them.

Still with me?

  // retrieve the customer
  Customer customer1Found = this.customerRepository.findByPropertyValue("firstName", FIRST_NAME);
  
  //
  // Tests
  //
  
  // can we find/retrieve the customer?
  assertNotNull("Unable to find customer.", customer1Found);
  
  // can we find the specific customer for which we are looking?
  assertEquals("Returned customer but not the one searched for.", FIRST_NAME, customer1Found.getFirstName());
  
  // does the retrieved customer have its non-indexed properties returned, as well?
  assertEquals("Returned customer doesn't have non-indexed properties returned.", LAST_NAME, customer1Found.getLastName());

  // retrieve the customer's purchases
  // (NOTE: We case as a Collection just to make checking the number of puchases easier)  
  Iterable purchasesIt = customer1Found.getPurchases();
  Collection purchases = IteratorUtil.asCollection(purchasesIt);
  
  // do we have the correct number of purchases?
  assertEquals("Number of purchases do not match.", 2, purchases.size());

So now we get to some actual testing.

The tests above are all straightforward.  We ensure the following:

1. We can retrieve a persisted node, specifically via an indexed property.
2. We can retrieve the correct persisted node for which we are searching.
3. We can view non-indexed properties from the retrieved node.
4. We can retrieve the correct number of relationships of the retrieved node.

As noted in Section 9.3 of "Good Relationships", we use Iterable for those node properties that are collections and are to be left as read-only, and Collection or Set for those collections that can be modified.

  // go through the actual purchases...
  Iterator purchIt = purchasesIt.iterator();
  Purchase purchase1 = purchIt.next();
  
  // retrieving objects via Spring Data pulls lazily by default; for eager mapping, use @Fetch (but be forewarned!)
  // ...this means we have to use the fetch() method to finish loading related objects
  Stock s1 = template.fetch(purchase1.getItem());

What if we want to view a node's related nodes' data?

By default, Spring Data loads an entity's relationships lazily, which makes perfect sense (just picture how much memory would be needed if you had a very large, highly connected graph).  Also remember that there are implicit relationships between entities if an entity is contained as a property of another entity.

(Courtesy of Paramount Picture's Forrest Gump)
"Mama said eager loading is like a box of chocolates: You never know what you're gonna get."
Well, at least the chocolates had an easily-determined, finite number in the box...

It is possible to have an eager retrieval by using the @Fetch annotation (be warned, though, that it will currently only work, by default, on node entities and collections of relationships that are based on Collection, Set, or Iterable; Spring Data may expand that in later releases, but I believe you can extend the mappings to work with other classes, if you so desire).

So, with our lazily-loaded relationships, we can use "template"'s fetch method to finish loading in the missing data.  It's as simple as that!  Anyone familiar with ORM will get this immediately.

  // can we retrieve our first purchase successfully w/ its details?
  assertEquals("Purchased item not persisted properly.", GAME_TITLE, s1.getTitle());

  purchase1 = purchIt.next();  
  Stock s2 = template.fetch(purchase1.getItem());
  
  // can we retrieve our second purchase successfully w/ its details?
  assertEquals("Purchased item not persisted properly.", GAME_TITLE_2, s2.getTitle());
  
  // if we're here, then all test ran succesfully.  Hooray!
 }

Above, we run a couple more tests to ensure that we can, in fact, retrieve and view lazily-loaded objects from Neo4j.

Nothing to it!

Making Friends the Easy Way: By Creating Them!
For these tests, we're going to have a look at something a bit more social, i.e. customers befriending other customers (how Utopian!).  I suppose we could make them "rivals" or "enemies", but that's a bit too sinister for this blog (for now...).

Anyway, I have prior to this test method a setup method (annotated with the @Before JUnit annotation) that creates 5 customers (if you're interested, I persist them using a CustomerRespoitory I created by extending the GraphRepository and RelationshipOperationsRepository interfaces).

 @Test
 public void customerFriends()
 {
  // add friends
  c1.addFriend(c2);
  c1.addFriend(c3);
  c1.addFriend(c4);
  c1.addFriend(c5);

  // be careful! setting a "Direction.BOTH" relationship in one node entity will have the ENTIRE relationship saved (*including the adjoining node*) when saving just ONE of the two entities!
  // ...if you save both, Neo4j will remove the duplication (and you'll be left wondering why c1 is a friend of c2, but not vice versa)
  
  // save c1's friends
  customerRepository.save(c1);

In the code above, we have the customer "c1" make friends with the other customers (he's a social butterfly).

Now, perhaps the most important part of this whole blog is shown here (and below).  It has to do with relationships, specifically those that are annotated as being "Direction.BOTH".

As you can see from the comments in the code above, we need to be careful about how we create relationships between nodes and save them.  If we were to create the relationship between, say "c1" and "c2", and then persist each node (and therefore the relationships, which the customer repository will handle), we would notice that the relationships have gone awry, and that the duplicate relationship from "c2" has been removed.

So, what we're going to do is the following (keeping in mind that "c1" and "c2" have already been persisted in the setup method):

1. Persist "c1" (and thereby its friendship to "c2").
2. Retrieve "c2".
3. Add any other friends' relationships to the retrieved "c2" (while not befriending back to "c1").
4. Persist "c2" (and thereby its friendships to those added in Step 3).

Step 1 is done above.

  // we can't just continue to add friends to this.c2, as once we try to save this.c2, it'll remove the duplicate relationship between c1 and c2.
  // ...so, to get around this, we retrieve the persisted object from the DB
  Customer c2Found = customerRepository.findByPropertyValue("lastName", C2_LNAME);
  c2Found.addFriend(c3);
  c2Found.addFriend(c4);
  c2Found.addFriend(c5);

  // save c2's friends, which will preserve the existing relationship with c1! Old friends can remain friends!
  customerRepository.save(c2Found);

As you can see above, we finish the remaining steps (2 through 4).

Again, note that we DO NOT create a reciprocal relationship from "c2" to "c1".  (One would hope a friendship relationship would be reciprocal; unless you have stalkers or something...)

This would totally help.

All that's left now is to run some tests to ensure that our friends have remained friends throughout all this persisting!

  // retrieve c1 for some tests
  Customer c1Found = customerRepository.findByPropertyValue("lastName", C1_LNAME);
  
  Iterable c1Friends = c1Found.getFriends();
  Collection c1FriendsSet = IteratorUtil.asCollection(c1Friends);
  Iterator custIt = c1Friends.iterator();
  
  int numFriends = 0;
  
  // let's make sure all of c1's friends were retrieved
  assertTrue("Friend not found.", c1FriendsSet.containsAll(IteratorUtil.asCollection(c1.getFriends())));
  
  // let's also make sure that c1 and c2 are still buds specifically (these two are inseparable...you should see them at ComicCon!)
  assertTrue("Friend not found.", c1FriendsSet.contains(c2));
  
  // let's make sure the exact number of friends returned is correct 
  while (custIt.hasNext())
  {   
   custIt.next();
   numFriends++;
  } // while
  
  assertEquals("Number of friends returned incorrect.", 4, numFriends);

  // if we're here, all is well! Huzzah!
}

Above, as in the first test, we make sure that all of the friendships have been properly preserved, both from "c1"'s and "c2"'s perspective.

Conclusion
In this post, we have seen the basics of persisting with Spring Data and a couple of the quirks I ran into.  These are documented within the Spring Data documentation, but it never hurts to bring these little nuances out into the light even further.

We also saw that the ImpermanentGraphDatabase is available to us through the Neo4j kernel which is a wonderful tool for implementing test cases with quick setup and teardown--no needing to write initializers and cleaners for a Neo4j installation!

So there we have it!  A first pass through persisting with Spring Data and implementing some unit tests using Neo4j and JUnit.

If anyone has any questions or I've made a mistake, please feel free to leave feedback.

We'll see you on the next post!

Supporting an Entrepreneur: Do You Have What It Takes?

When most people think about entering the workforce (or shuffling around a bit in it), they typically think of two types of employment:

1. "I'm going to work a 9-to-5 job, collect my paycheque every x weeks, do a decent job, work some overtime, earn a promotion or two, keep my work life very detached, and make sure I have plenty of time to do what I want."
2. "Screw working for other people--I'm going to start up my own company, be my own boss, and make a bunch of money.  Sure, I'll have to sacrifice most/all of my own time and figure out a way to fund my venture, but I know I can make a difference and be guaranteed to do something I love."

In other words, type 1 is the "9-to-5er" while type 2 is the "entrepreneur" (and, depending on circumstances, this can also apply to freelancing).

That would make for a good movie title: "Arnold Schwarzenegger is...The Entrepreneur!"

In this article, we're going to have a look at a third option which I find is largely overlooked (granted, I'm sure there are far more options/goals for employment out there, but in the interest in keeping this simple, I'm sticking to the ones in this article).

This third option is what I refer to as the "entrepreneur supporter".

I had originally planned on writing another series on this matter; however, I'm going to stick to a single article on it and, if interest/time allows, will expand upon the ideas in here in subsequent posts.

Disclaimer

It is important to note, as well, that I am neither advocating nor denouncing one form of employment or the other (or the people who choose them).  What I am doing is writing from my own experience; take it for what it's worth, i.e. caveat emptor).

Decisions, Decisions...

Let's have a look at the first two types of employment and consider what they tend to mean to people.

What we will find is that a lot of the pros and cons are really a matter of perspective, so what one person may find to be a con may be a pro to another.  Keep that in mind as you read through this article.

The 9-to-5er

Pros:

1. Easier to keep work/life balance and, as The Offspring said, keep them separated.
2. Fewer concerns about collecting a regular paycheque.
3. No risks involving your own money.
4. "Work to live", not "live to work".

Cons:

1. For the ambitious, limited control/say in how to grow/steer the company.
2. More likely to be out of the loop as to the status of the company (is it in financial trouble? etc.).
3. It's easy to lapse into a sense of entitlement (further reading to help combat this within your organization: Ownership Thinking by Brad Hams--highly recommended).
4. Likely to have your salary/position cap out somewhere not too much higher than where you are now (of course there are exceptions, and for some people, this is less a con than a pro; the lesson: it's important to know what you want).
5. Working hours are laid out by employer (but seem to be more flexible these days).

For a lot of people, this type of employment is exactly what they want; for others--especially the ambitious--they want something more (or, at least, different).

Let's have a look at the next type.

The Entrepreneur

Pros:

1. Don't report/answer to anyone but yourself (well, at least until you have a Board and/or shareholders).
2. Set your own hours/schedule.
3. You determine exactly what kind of work you want to do (and it's likely something you love--otherwise, why do it?).
4. You determine what you do with all revenue.

Cons:

1. Have to fund the venture yourself (or at least raise the funds).
2. It's your skin in the game: If business is poor/suffering, it's your money on the line.  Not to mention having to cover myriad costs...
3. All the headaches taken care of by managers, operations, executives, sales, etc. are now your problem.  (Read: Insomnia.)
4. Frustration.  You'll have to generate new leads/sales all while you're doing the work you set out to do (unless you hire someone else to help with that, but that's yet another cost to bear).
5. While you set your own schedule, in order to get the business machine going, you're unlikely to have very much free time at all (if any) for the foreseeable future.
6. Taxes, taxes, taxes.
7. More taxes.
8. ...and that's all while you actually do the work you set out to to do!

For those who are ambitious and want this kind of life, it can be extremely rewarding; of course, it's also fraught with risk and peril (kind of like being a knight, but with less chance of being parodied).  The greater the risk, though, the greater the reward.

Like you didn't see this reference coming a mile away.

Surely There Must Be More

So what about those who are looking for something in between those two options?  Something more, where one can affect greater change without needing to commit the shirt off one's back.

There is more, and don't call me Shirley (sorry, couldn't resist).

(From Paramount's Airplane)

Ok, no more Airplane jokes--I promise!

A Bit of Background

Being an "entrepreneur supporter" is an area in which I have considerable experience.

Coming out of school, I had the opportunity to work for various companies as part of my degree's co-operative program.  The companies for which I worked ran the gamut of SME sizes.  I also worked for large, multi-billion dollar corporations (that sounds impressive when I put it like that).

Needless to say, I had a taste of what was involved in working for each size of company.

Without going into gory detail, I had decided that while I wasn't prepared to risk everything, I had more ambition than struggling my way up the corporate ladder for the rest of my natural life, relying on luck and politics as much as hard work and skill in order to get ahead.

There's ambition, and then there's ambition.

As I couldn't really find any direction/advice on what to do in this case (save for some advice which I'll just say was trite), I thought about it, did my own research, and came up with a plan to try out:

I would get a job with a smaller SME in which I could add immediate value, establish myself, invest a good chunk of extra time, learn what I needed to, and work my way to a position of influence as fast as possible.

It worked.

Yes, I had to invest more than a few years' worth of time before I had any kind of measurable influence (including huge swaths of my own), but it worked (by way of my own example, in a company of 40-50 people, I had made VP before I was 28 and currently sit as the CIO of the company).

No one said it would be easy.

Being an Entrepreneur Supporter

At this point, I think it's safe to say that being in such a position is a compromise between being a 9-to-5er and a full-on entrepreneur.

I'm going to share with you, dear reader, some of what I've learned during the course of being an entrepreneur supporter (I like it when section titles line-up so nicely!).

When choosing the company you want to work for, consider the following:

1. Make sure they do something you are actually interested in.  (Seriously.)
2. Do your due diligence!  Check out the owner and/or executive (if there is one), what their goals are, and what their short-term, medium-term, and long-term goals (and, if applicable, their exit strategy).
3. Make sure it's a size you're ok with.  As a general rule, the bigger the company, the more work (and time) you're going to have to put in (more politics, more inertia, etc.).
4. Make sure you can work with the entrepreneur/executive (let's face it: egos or not, entrepreneurs can be very particular about how they want things done; after all, that's why they started a business instead of working for someone else).
5. Do your homework on the company!

Strike up a relationship with the owner (and any of his/her executive staff) early on.  If you're going to help steer his/her company, as noted above, you'd best be able to work with that person.  What's more, you'll likely be spending a lot of time with him/her, so getting to know the owner/exec staff early on will go a long way, both in terms of being able to work with the people in question as well as understanding the business in which you're now embroiled.

Keep your ears open and ask questions.  Your MBA might have taught you a lot, or you might be great with technical jobs, but in a small business, chances are very, very good that you will not have learned enough to wear all the different hats you need to (whether you're responsible for those positions or not).  And the odds are good--especially if the business has been around for awhile--that the people currently working there know a thing or three about how to run it well.  Listen to them.  And if you think you know how to do what they do, challenge yourself and find out.  Grow.

Be prepared to be on-call 24/7.  Though most 9-to-5ers are now expected to be reachable for a bit after business hours and deal with occasional emergencies, being on-call in this supporting role is inevitable.  If you want people to listen to you, you have to participate, and that means late-night phone calls, early-morning meetings, last-minute conference calls, and emergencies on long weekends (and being responsive to all of the above).  The idea here is to make the entrepreneur's life easier--not yours.  He/she has enough to worry about.  Let him/her know you can handle the tough situations.  Start small, and the responsibilities will pile up (a mixed blessing to be sure).  It's a small price to pay for having influence without having to worry about your own money in the company.  Take comfort in knowing that you won't have to work 80 hours a week forever.

This will be you, for any number of reasons.  You've been warned.

Don't be the squeaky wheel.  Whatever you negotiate your starting salary/terms to be, make sure you're comfortable with it, because the last thing your boss will want to hear is you justifying why you should get a raise every 6-12 months.  Sure, you might get it every so often, but, that kind of sacrifice (or lack thereof) won't soon be forgotten.  Remember: He/she is sacrificing a lot more than you are; let them know they aren't alone in it and that you're "investing" in your own way.  If you've done your research, your entrepreneurial boss will take care of you (time off, flexible hours, pay raises when possible, etc.).  This actually leads into the next point nicely.

Find a shop where egos are checked at the door--including the entrepreneur's.  This doesn't mean the owner will be cow-towing to you and giving in to your every whim.  What it means is that everyone--including the owner--is prepared to do whatever it takes to make the business work.  You still have to deal with the owner's personality (like any boss), but at least you'll have an added measure of reassurance that you're not going to get taken to the cleaners and/or taken advantage of.

Treat company money like it is your own.  If you're in a position where you make decisions and/or handle company money, treat the money as though it was your own (and I mean that in the best possible way; i.e. don't go spending it all because you can).  Be smart with the money going out.  Be prepared to justify expenditures ("ROI" will become your favourite TLA).  Conversely, be prepared to do whatever it takes to maximize revenue.  Nothing makes you look better than adding serious value to the company's bottom line (an axiom in any-sized business).

Two words: Be flexible.  If you were hired on to be an engineer or developer, be ok with being pushed out of your comfort zone.  Take on tasks that you'll likely make mistakes with (just be careful with how you manage those mistakes--another good lesson).  If your boss wants you to go in his stead somewhere, do it.  If your boss wants you to take out the garbage, heck, do it.  Show that you're reliable.  Wax on, wax off (and all that).  And remember the Golden Rule: "He who has the gold makes the rules." Be prepared to do what the person signing your paycheque wants/needs.  The Golden Rule also speaks to what your boss does with his/her time.  You might not approve/believe it fair/whatever of what your boss does.  Guess what?  It doesn't matter.  It's the same with any job/boss, really, but at least here you can take solace in the fact that it is that person's money paying your wages--that person certainly isn't going to just burn money for the sake of burning money (see above: due diligence).  Get over yourself and focus on your job.  If it really bothers you, then maybe this type of work isn't for you.

Work/life balance.  This one might be a deal-breaker for many.  If you can manage to balance the two the way you like, then all the power to you.  However, be prepared and don't be surprised if those two areas of your life bleed over into each other constantly (and usually much moreso the "work" portion entering into your "personal life" portion).  This "bleed" is happening more and more with many jobs, but it's a fact of life with this type of work.  Entrepreneurs are notorious for having to sacrifice personal relationships for their dreams of their business.  Supporting those entrepreneurs will most definitely require a comparable level of sacrifice; maybe not to the same degree, but make no mistake: Sacrifice will be involved (I really can't overstate that).

Produce!  

No, not THAT kind of produce.

It should go without saying that you should produce results, no matter what tasks you undertake.  You'll likely have to take it a step further, too, and be prepared to do whatever it takes to complete your tasks with a minimum of budget.  Does your solution involve sinking $10 000 in licensing?  Or could you do a bit more work and find a way to cut that cost out with open source or a different solution?  This does not just apply to software, either.

It's still a big risk.  While you might not have skin in the game (unless you really want to go that route), you're certainly not a 9-to-5er.  As time goes on, you'll be exposed to more and more information--both good and bad--and you'll be expected to know what to do with it.  What's more, there's no guarantee that you'll see a big payday/cash out, but you're likely to experience more freedom (with your responsibilities), and with no risk, there's certainly no reward.  Make peace with that as soon as possibly can, and learn from any failures you may encounter.  "Success has many fathers; failure is the bastard son."

Round-up

I'm sure I could go on at greater length, but, what I've disseminated here is at the core of what it takes to successfully support an entrepreneur.

While there certainly is a greater degree of potential reward, as well as freedom and responsibility, it has its share of risks, as well, though not necessarily as severe as if you had started your own business.

I have to say, though, for me, at least, it's been well worth the risk.  Personally, I've invested a lot of my own time into supporting an entrepreneur, and it has taken a lot of sacrifice (it still does), but it can and does pay off.  I enjoy a considerable degree of freedom (along with the Sword of Damacles that often dangles above my head), as well as a long-time friendship with my boss.  It's definitely not for everyone, but for those who do pursue it, it can pay off huge.

I might write more about it at a later date, but in the meantime, questions and constructive feedback are most definitely welcome.

We'll see you on the next post!

Modern Consulting vs. Classic Consulting: Part 5 (Summary and Conclusion)

Ok, folks, we're down to the summary and conclusion of my series comparing classic consulting and modern consulting by way of our little thought experiment.

In this post, I'll be summarizing what we spent the last three posts examining and bringing things back full circle to the post that started this series (it's kind of like a flashback episode but without the laugh track).

No need for this today!

Let's start off with a little section I like to call...

Recap
Way back in the original post of this series, I'd stated that the company for which I work, BHS Consultants, is an "IT/technology consulting firm"; I'd also stated that this label is a "challenging" one.

By way of comparing and contrasting what I refer to as classic consulting and modern consulting, I set out to define what I meant by the two respective terms with the goal of showing why the label above is a challenging one for my company (which I will finalize in this post).

We compared the two terms by way of a thought experiment, defined as follows:

1. You are in charge of a large project to implement a new business process that is to be heavily automated.
2. You have a fixed budget and deadline.
3. You currently do not have the expertise or staff to undertake and complete the project.
4. You want to make as big (read: positive) an impression as possible.

We then considered three different options to tackle this project which can be summarized as follows:

1. Find and take on an internal staff (either full-time or contract) including both SMEs (Subject Matter Experts) and implementors.
2. Find and contract a knowledgeable consultant to advise on the project and then either field a staff to implement or contract out the work to another firm.
3. Find and contract a knowledgeable "one-stop shop" that has both the SMEs needed and the manpower to carry out the plan/advice from the SME(s).

Each option is linked below for convenience (I know, I know; I'm just that nice).

Option 1

Option 2

Option 3

Bringing Things Back Full Circle

So now we've seen three different options to address the same situation, all of which have their places.

However, in my experience, when it comes to consulting on technical matters (e.g. as in our thought experiment involving heavy automation)--especially where software and programming is involved--the third option is the one that makes the most sense, both from a time and cost perspective.

Some of you may argue that the above model failed horribly with the outsourcing fiascoes of the 2000s (I'm sure some of you had good experiences, but from my own experience and others I've dealt with, many did not), and that's a topic for another post.  It's difficult to argue with numbers that help your bottom line tremendously ($5/hour for 10 developers? that's tough to beat); however, you get what you pay for, and in many cases, it was either nothing or a product/service of inferior quality.

(Again, I'm not here to bash on every outsourcing venture, but am merely drawing from my own experiences and knowledge.)

But I've also witnessed (and been a part of) many successes with that same model with firms in North America/Europe (some would call it near-sourcing).  When companies see the value we bring with the deliverables we produce, it sinks in with them, and they become hungry for more, which we're only too happy to provide.

So What about That "Challenging Label"?

So why is being called an "IT/technology consulting firm" a challenging label for us?

Because, many times, it sounds like the company is a part of option number two, i.e. in the classic consulting space.  By that definition, BHS Consultants is definitely not a "consulting" firm; however, if you consider option number three, i.e. the modern consulting space, we most certainly do fit the bill.

Explaining those differences and trying to describe the services we can provide is perhaps the most challenging issue we face.  We certainly can't be overly esoteric about what we do, but to list the myriad skills and services we bring to the table would be an invitation to read through a novel!

Can you imagine this as a website?  Yeah, me neither.

Regardless of the challenges, though, we seek to add as much value to our clients as humanly possible.  BHS is extremely passionate about what they do, and I'm no exception. I wouldn't have spent the past decade plus here if I didn't absolutely love what I do.

And while "value" is usually in the form of deliverables (be they tangible or not), we always seek to help educate our clients (and even prospective clients) on the services we provide and the matters in to which they are delving.

Empowering clients to make smart, informed decisions about technology is at the heart of BHS Consulants, and should be at the heart of any IT/technology consulting firm; but being able to help clients actually realize those decisions is where the real value comes in to play.

Conclusion

Ok, there you have it!  We are finally through the entirety of this series!

Thank you for making it this far and reading through (hopefully) all of this series.  I do hope it's been insightful with respect to what I/we do, and with respect to the state of modern consulting.

Please do check us out at bhs-consultants.com (shameless plug, but hey, it's relevant!).

Remember: Both classic consulting and modern consulting have their roles to play in the right situation; but with the ever-expanding role technology plays in day-to-day operations of businesses and the ever-increasing request for plans and advice to be executed and realized, modern consulting will continue to become more and more relevant (and prevalent) by the day.

Thanks once more for your time and patience!

I'll see you on the next post!