We've released a new version recently. Here are the changes we have made:
[main] Added constraint option allowing to skip the constraint if the field isn't changed
[main] DifferentialTuple.Merge() operation no longer causes StackOverflow exception
[main] Fixed certain cases of foreign key absence for reference fields of structures
[main] Fixed translation of enum value as default field value
[sqlserver] Fixed extraction of information about indexes from database
As usual you can download it from our official website or install it from Nuget library
Complete changes description is below.
I'd like to share an early link to feature-based ORM comparison we're working on: "The Most Comprehensive Feature-Based Object-Relational Mapping Tool Comparison Ever :)™"
The document is incomplete yet:
- some cells are empty - i.e. their content is currently unknown;
- there can be some mistakes (it wasn't checked by community yet);
as you might suspect, a copy of this document is edited by ORMBattle.net participants, so its version including most of the tools tested there must also appear soon. It won't appear "as-is" at our own web site, but we'll use ~ the same columns from it (direct comparison with commercial competitors in marketing materials is normally not acceptable).
On the other hand, the feature map is already quite comprehensive: there are about 270 features organized into hierarchical structure. It is far more detailed then any other ORM comparison we were able to find (likely, this one is the most detailed, but really ancient predecessor).
Likely, the document is currently a bit biased toward DataObjects.Net from the point of selected features, but I feel this will be "automatically fixed" by the community shortly: vendors are allowed to add any non-duplicating features and sections there, as well as propose to exclude the non-important ones.
On the other hand, it's clearly much less biased document as e.g. this one (although I understand it doesn't pretend to be a real comparison). I.e. it can be hardly called as promotional material.
Our final goal is to develop a feature map including major ORM tools and features that are mutually agreed by various ORM vendors, where each vendor is responsible for contents of his own column (i.e. cheating is possible, but I suspect users & competitors won't accept this well); the table you see is our initial investment into this process.
Availability of such comparison should help developers to choose the tools they need based on their own requirements, as well as understand the relationships between features better (hierarchy seems really helpful here - I already got few quite positive comments related to the structure of the document).
An accompanying document commenting each section there and describing DataObjects.Net advantages / disadvantages in comparison to other tools should also appear soon.
Don't forget to study the comments at the bottom of the first sheet, as well as "Remarks" sheet.
This post actually starts a sequence of posts dedicated to DataObjects.Net design - more precisely, to its design goals. I decided to start the cycle from the practical example, since picture frequently worth more then thousands of words. Certain amount of advertisement of our outsourcing team is just side effect of this post, although if have some serious project for these guys, you're welcome.
SeverRegionGaz is regional natural gas provider. This is a big organization, that, although being a subdivision of Gazprom itself, has a set of branches as well. There is a bunch of legacy software systems there, varying from pretty old to new. Some of them have very similar functionality: earlier SeverRegionGaz branches were independent from each other, and thus they use partially equivalent software.
Most of data they maintain is related to:
- Billing - obviously.
- Equipment. E.g. they know exactly what's installed at each particular location (home, office, etc.).
- Incidents and customer interaction history.
- Bookkeeping. Unfortunately for us, they wanted to see certain information from two different instances of 1C Enterprise 7.5 there as well.
The main goal of "Single Window" system is to provide a single access point allowing to browse all this data, and, importantly, search for any piece of information there.
Let me illustrate the importance of this goal: earlier, to find necessary piece of information (e.g. billing and interaction history for a particular person), they should identify one or few of these legacy systems first, and then request necessary information from appropriate people (nearly no one precisely knows all these systems). In short, getting the information was really complex and long process -- and we were happy to change this.
There were few other, minor goals - e.g. it was necessary to:
- Provide web interface allowing customers to report the values of natural gas consumption counters and interact with support staff.
- Integrate with external payment processing system provided by their bank and automatically process the payments made by customers.
- Implement reporting. Only a part of reports needed by SeverRegionGaz was available in legacy systems, so we should implement the missing ones.
To stress this, we should develop an application allowing to browse really huge database (I'll explain this later). Its editing capabilities should be pretty limited - mainly, because:
- Most of this information must be imported from external sources. If we'd be asked to support editing, it would bring the complexity to a completely different level. In fact, we should either be capable of syncing back all the changed (that's really hard, taking into account that none of legacy systems is ready for syncing, and almost no one knows how these systems exactly work at all), or, alternatively, replace all the legacy systems there by our own (hard as well).
- That fact that different systems are still necessary for editing (mainly, data entry) is acceptable for SeverRegionGaz. The people working with them are used to them; single person there normally deals with a single system. Having "Single Window" there, they should study just one more system to be capable of accessing all the data - that's much better then ~10.
- Full editing support (likely, a complete replacement of most of legacy systems) is primary goal for "Single Window v2" - and the idea of movement to this big goal step-by-step is really good. For now it's ok if we allow to edit only the data we fully control.
That was a story behind "Single Window" project. Now some facts about its implementation:
- Time: 7 months -- February 2010 ... July 2010. First 1.5 months were spent almost completely on specifications.
- People: initially -- 3.5 developers + 1.5 managers (Alex Ustinov was playing both roles there); closer to completion -- 6 developers.
- Database: ~ 12 GB of data, 507 tables, 440 types! (all are unique, i.e. there are no generic-based tables)
- External data sources: 8, full data import is implemented for all of them; in additional, continuous change migration is implemented for 3 of them.
- Other elements: hundreds of lists, forms and reports. I suspect, totally - almost 1 thousand.
- Complexities: lots of, but mostly they were related to ETL processes, starting from some funny ones and ending up with real problems.
- Used technologies:
- DataObjects.Net 4 - btw, it's our first really big application based on it. And that's why I write this post :)
- LiveUI - it would be really hard to generate that huge UI without this framework. Btw, Alex Ilyin adopted its core part to WPF pretty fast. We also intensively used T4 to
- WCF - we've implemented 3-tier architecture relying on WCF as communication protocol.
- WPF - an obvious choice for UI,
and lots of other stuff, ending up with pretty exotic COM+ (used for integration with 1C).
Screenshots
The main tab is designed in very minimalistic fashion:
That's what happens when user hits "Search" button:
As you see, we use full-text capabilities of DO4 in full power here. In fact, we index all of objects, which content is interesting from the point of search. Full-text indexing here is implemented in nearly the same way as it was in v3.9 - i.e. there is a single special type for full text document, per-type full-text content extractors running continuously in background, and so on.
Here is a typical list (take a look at grid settings and search box):
Some forms:
The list of actions in left panel is actually pretty long - i.e. you see may be 30% of it. There is no scrollbar, but you may find "Up" and "Down" arrows indicating the list will automatically scroll up or down when mouse pointer approaches its top and bottom edges.
Integration services control list:
Web site, customer's home page:
Pages for registration of natural gas consumption counter value and interaction with support staff:
And finally, a single screenshot exposing the complexity of domain model:
As you see, our team made a huge job, that is directly related to DataObjects.Net.
The main point of this post is: DataObjects.Net is designed to develop really complex business applications fast. Why? Well, that's the topic for my subsequent posts, but for now I'd like to touch key points:
- Code-only approach allows developers to work fully independently without caring about schema changes at all - even in different branches.
- Integrated schema upgrade capabilities are ideal for unit testing (and testing in general). It's easy to launch unit tests for any part of your application.
- Rich event and interaction model simplifies development of shared logic, such as full-text indexing and change tracking.
- Excellent LINQ support brings significant advantages, when you start writing reports. Queries there might require really good translator.
- Can you imagine dealing with 500+ types in EF? Actually, I just tried to find some reports about this on the web, but found only "avoid this" statements, with tons of reasons. The most funny one is about performance and usability of IntelliSence when you type "dataContext.".
To be frank, of course I got lots of issue reports during these months from our "Single Window" team, and actually, still get them. Mostly they were related to LINQ and schema upgrade. So if you use DO, you can say "thanks" to Alex Ilyin - he simply tortured me and Alexis Kochetov, and still does this. E.g. mainly because of him:
- DO4 translates really complex LINQ expressions involving DTOs (custom types and anonymous typs).
- Schema upgrade works really well and fast now. Domain.Build(...) performance is 2-3 times higher there (Alex hates waiting for launch). E.g. now our 440-type Domain requires ~ 4 seconds to be built in Skip mode on my moderate office PC (Core 2 Duo). To achieve this, I should parallelize some stages of build process.
Let me finish with one more screenshot:
P.S. I'm really interested in examples of complex applications relying on popular ORM tools. If you know some, please share the link. I got an impression that people still avoid using ORM tools in such cases (there are opinions like "ORM is not for enterprise!"). So I'd like to "measure the length" in terms of model complexity (count of tables, types, etc.) - just for fun, of course. You should know we like to measure various features of ORM tools.
There is a great post by Ayende: "The false myth of encapsulating data access in the DAL". I'm posting the link to this article not just because I fully agree with its main thesis, but also because I know this is sitting in people's brain quite deeply.
I quite frequently get questions like "how can I design my DAL so that it will require nearly zero efforts to replace it?", or "how must I use DataObjects.Net that way?". And it's always quite difficult to make the people believe it's simply a bad idea, that will lead to exaggerated complexity and finally, waste of money.
Worse, our outsourcing department even have a customer, that put this requirement into SRS: "it should be easy to migrate from EF to any other DAL in that project". IMHO (sorry, Denis :) ), the only effect satisfying this requirement is noticeably more complex implementation in pretty simple project. Btw, EF was chosen for this project also mainly because of that requirement: EF is probably the less featured framework from any point except mapping. In the end of all we implemented this requirement, but got nearly 25% delay in delivery. Since I estimate the chance of migration to another ORM as something like 1%, this is nothing but just money waste at our side.
Speaking differently, all these people want to have ORM abstraction layer in their application. For ORM developer like me this sounds a bit ridiculous: ORM itself is designed to be an abstraction layer from the database, so finally these people want to develop an abstraction layer for another abstraction layer.
Why? The most frequent answer is: "I'd like to have an opportunity to switch to another ORM in future.". That's great, but it seems most of such people don't understand that complexity of developing such layer is comparable e.g. to complexity of such libraries as Qt. Do you agree GUI libraries are pretty similar? Initially it looks like true: there are windows, controls, etc., and most of them are nearly identical (scrollbars, buttons, ...). But anyone who knows all the details will say "it's hell, they're quite different!".
Of course, analogy between GUI and ORM tools may look strange: GUI tools are developing during 30+ years, but ORM tools started their way of wide adoption just 5...10 years ago. On the other hand, they're getting more and more complex. Compare what you can do now (LINQ) with ORM tools we have 5-7 years ago. This implies it will be harder and harder to develop ORM abstraction layer in future, because they'll evolve further nearly as GUIs and other well-known APIs.
So why people believe so much in ORM abstraction layer?
I feel this happens mainly because almost any book describing enterprise application design patterns makes a false statement in some form, and moreover, proves by relatively simple, and thus completely unrealistic example. Moreover, people tend to think even statement like "UI, BLL and DAL must be clearly separated" means "you must abstract your DAL from a particular ORM"!
And that's why, before thinking about developing your ORM abstraction layer, I strongly recommend you to:
- Read this post by Ayende: "The false myth of encapsulating data access in the DAL".
- Try to name at least one publicly recognized application having integrated ORM abstraction layer. To be honest, I don't know such applications. If you do, please name them in comments below this post.
So likely, you can't imagine my happiness :) Ayende is a very famous guy, so it's great he wrote about this problem. I really tired feeling myself fighting with this myth alone. Of course, I wasn't, but you know, it's hard to prove something while you don't rely on facts said by widely recognized people. And in this case my feeling was opposite: it was looking like nearly everyone recommends to design your application without binding to a particular ORM.
P.S. I also noticed a very good comment to that article: "Along the same line, you should not embrace a n-tier architecture when you can just build a 2-tier app. 2-tier can be a lot easier if you accept the limitations, and easier = time to market, cost, maintainability = money.". This fits quite well to our experience with our EF-based project I wrote about. It was designed as 3-tier application, although IMHO it was absolutely unnecessary in this case. So again, that's one more decision exaggerating the complexity and cost. So I feel I must be happy we got just 25% delay in this case :)
|
|
