News, examples, tips, ideas and plans.
Thoughts around ORM, .NET and SQL databases.

Wednesday, July 29, 2009

Preliminary ORM performance comparison: DataObjects.Net 4 vs NHibernate

We've just adopted our CrudTest for NHibernate. First results:

DO4 (LINQ):
Insert: 28,617 K/s.
Update: 34,111 K/s.
Fetch & GetField: 8,682 K/s.
Query: 1,486 K/s.
CachedQuery: 8,176 K/s.
Materialize: 358,671 K/s.
Remove: 41,108 K/s.

NHibernate (LINQ):
Insert: 12,936 K/s.
Update: 12,939 K/s.
Fetch & GetField: 7,152 K/s.
Query: 95,7/s.
CachedQuery: Cached queries are not supported in NH yet.
Materialize: 37,892 K/s.
Remove: 13,012 K/s

The highlighted numbers show ~ 10 times difference, although DO wins in all other cases as well. Results of this test for several other ORMs are upcoming, the project will be shared @ Google Code.

In addition, we're developing general LINQ test as well. For now LINQ for NHibernate passes ~ 25 tests out of 100. This mean only very basic LINQ stuff really works on this release. DO4 passes ~ 98 tests there (it still misses passing arrays/collections as query parameters).

P.S. The optimization we've made during last 2 weeks is already quite successful (although we still work on materialization performance). Pre-optimization results can be found here.

4 comments:

  1. Very impressive. Well done, guys.

    ReplyDelete
  2. of course you missed the price / performance statistics.

    ReplyDelete
  3. Err... Is it normally provided in any other performance comparison? Is there a single performance score in our tests we can use? What should be done with differently priced editions of each product?

    In short, price / performance is actually a very subjective number.

    IMO, even this question is more meaningful: let's say you get 30% better performance for X$ (fixed amount of money) for your Windows Azure application requiring you to pay C$ / month in average. How much time would it take to return the initial investment of X$?

    ReplyDelete