I quite frequently get questions like "how can I design my DAL so that it will require nearly zero efforts to replace it?", or "how must I use DataObjects.Net that way?". And it's always quite difficult to make the people believe it's simply a bad idea, that will lead to exaggerated complexity and finally, waste of money.
Worse, our outsourcing department even have a customer, that put this requirement into SRS: "it should be easy to migrate from EF to any other DAL in that project". IMHO (sorry, Denis :) ), the only effect satisfying this requirement is noticeably more complex implementation in pretty simple project. Btw, EF was chosen for this project also mainly because of that requirement: EF is probably the less featured framework from any point except mapping. In the end of all we implemented this requirement, but got nearly 25% delay in delivery. Since I estimate the chance of migration to another ORM as something like 1%, this is nothing but just money waste at our side.
Speaking differently, all these people want to have ORM abstraction layer in their application. For ORM developer like me this sounds a bit ridiculous: ORM itself is designed to be an abstraction layer from the database, so finally these people want to develop an abstraction layer for another abstraction layer.
Why? The most frequent answer is: "I'd like to have an opportunity to switch to another ORM in future.". That's great, but it seems most of such people don't understand that complexity of developing such layer is comparable e.g. to complexity of such libraries as Qt. Do you agree GUI libraries are pretty similar? Initially it looks like true: there are windows, controls, etc., and most of them are nearly identical (scrollbars, buttons, ...). But anyone who knows all the details will say "it's hell, they're quite different!".
Of course, analogy between GUI and ORM tools may look strange: GUI tools are developing during 30+ years, but ORM tools started their way of wide adoption just 5...10 years ago. On the other hand, they're getting more and more complex. Compare what you can do now (LINQ) with ORM tools we have 5-7 years ago. This implies it will be harder and harder to develop ORM abstraction layer in future, because they'll evolve further nearly as GUIs and other well-known APIs.
So why people believe so much in ORM abstraction layer?
I feel this happens mainly because almost any book describing enterprise application design patterns makes a false statement in some form, and moreover, proves by relatively simple, and thus completely unrealistic example. Moreover, people tend to think even statement like "UI, BLL and DAL must be clearly separated" means "you must abstract your DAL from a particular ORM"!
And that's why, before thinking about developing your ORM abstraction layer, I strongly recommend you to:
- Read this post by Ayende: "The false myth of encapsulating data access in the DAL".
- Try to name at least one publicly recognized application having integrated ORM abstraction layer. To be honest, I don't know such applications. If you do, please name them in comments below this post.
So likely, you can't imagine my happiness :) Ayende is a very famous guy, so it's great he wrote about this problem. I really tired feeling myself fighting with this myth alone. Of course, I wasn't, but you know, it's hard to prove something while you don't rely on facts said by widely recognized people. And in this case my feeling was opposite: it was looking like nearly everyone recommends to design your application without binding to a particular ORM.
P.S. I also noticed a very good comment to that article: "Along the same line, you should not embrace a n-tier architecture when you can just build a 2-tier app. 2-tier can be a lot easier if you accept the limitations, and easier = time to market, cost, maintainability = money.". This fits quite well to our experience with our EF-based project I wrote about. It was designed as 3-tier application, although IMHO it was absolutely unnecessary in this case. So again, that's one more decision exaggerating the complexity and cost. So I feel I must be happy we got just 25% delay in this case :)
P.S. I also noticed a very good comment to that article: "Along the same line, you should not embrace a n-tier architecture when you can just build a 2-tier app. 2-tier can be a lot easier if you accept the limitations, and easier = time to market, cost, maintainability = money.". This fits quite well to our experience with our EF-based project I wrote about. It was designed as 3-tier application, although IMHO it was absolutely unnecessary in this case. So again, that's one more decision exaggerating the complexity and cost. So I feel I must be happy we got just 25% delay in this case :)
Very good article ("The false myth of encapsulating data access in the DAL"), and also good blog post Alex :-)
ReplyDeleteI agree with all that was written. Even my friend (sorry Tomas) trying to encapsulate DataObjects framework into DAL because he want to have possibility to remove DO4 anytime and replace it with other ORM in future. And i dont have arguments for him not to do that. Now i have 1 and must hope he accept it.
Hello, I generally agree with what has been said, but there are some points...
ReplyDelete>> so finally these people want to develop an >>abstraction layer for another abstraction layer.
All programming is a building of abstractions above abstractions, so there is nothing fun about it.
>>EF is probably the less featured framework >>from any point except mapping.
That's too strong a statment. It might be Your point of view, but I do not think it is generally accepted as you state it.
And finally,
>> "it should be easy to migrate from EF to any >> other DAL in that project".
I find it absoultely sensible. It does NOT mean there should be some abstraction level over ORM. Probably, there is no such a thing in a project you mentioned. So, I suppose, there is no additional cost really. By the way, 25% of schedule is better than average in the industry.
> All programming is a building of abstractions above abstractions, so there is nothing fun about it.
ReplyDeleteIt becomes fun when effect/cost metric is taken into account.
> That's too strong a statment. It might be your point of view, but I do not think it is generally accepted as you state it.
Yes, that's probable. I was talking about our own case (we were thinking about DO and NH as well), and in this case EF was preferable choice: at least, if we'd start from DO or NH, likely, it would be harder to support EF as well.
Another thing is that its behavior set was, probably, less known to us in comparison to DO and NH. So it was a reasonable choice.
> I find it absoultely sensible.
Well, I think differently. Either you use the framework in full power relying on all its advantages as it's recommended, or invent some plumbing stuff to ensure "it's easy to migrate some day".
Frankly speaking, I think the statement must be even stronger: forget about migration to another ORM at all. If this will be necessary, most likely it will be cheaper to port the application to another ORM rather then constantly waste efforts on something that will be necessary with 1% probability.
I'd say it's reasonable to take into account features you'll need with may be 10% probability during the next year, although even this is very opposite to agile approach, where you must be focused only what you really need. But 1% in several years is actually too small number to even think about.
The analogy I see is building a house taking into account the fact that some day we may need to replace it on another foundation. Of course, this may be a bit exaggerated pucture, there is definitely some sence.
I have another good example of this in our company: MEScontrol.net. It works. It still relies on DO v3.9. The complexity there is simply incredible in comparison to most of systems. And, what's more important, the people developing it never tried to build similar abstraction layers.
Frankly speaking, that's one of best features of Aexey Belov and Alexey Filatov characters. They do exactly what they need, and nothing more. That's what agile really implies.
> All programming is a building of abstractions above abstractions, so there is nothing fun about it.
ReplyDeleteIt becomes fun when effect/cost metric is taken into account.
> That's too strong a statment. It might be your point of view, but I do not think it is generally accepted as you state it.
Yes, that's probable. I was talking about our own case (we were thinking about DO and NH as well), and in this case EF was preferable choice: at least, if we'd start from DO or NH, likely, it would be harder to support EF as well.
Another thing is that its behavior set was, probably, less known to us in comparison to DO and NH. So it was a reasonable choice.
> I find it absoultely sensible.
Well, I think differently. Either you use the framework in full power relying on all its advantages as it's recommended, or invent some plumbing stuff to ensure "it's easy to migrate some day".
Frankly speaking, I think the statement must be even stronger: forget about migration to another ORM at all. If this will be necessary, most likely it will be cheaper to port the application to another ORM rather then constantly waste efforts on something that will be necessary with 1% probability.
I'd say it's reasonable to take into account features you'll need with may be 10% probability during the next year, although even this is very opposite to agile approach, where you must be focused only what you really need. But 1% in several years is actually too small number to even think about.
The analogy I see is building a house taking into account the fact that some day we may need to replace it on another foundation. Of course, this may be a bit exaggerated pucture, there is definitely some sence.
I have another good example of this in our company: MEScontrol.net. It works. It still relies on DO v3.9. The complexity there is simply incredible in comparison to most of systems. And, what's more important, the people developing it never tried to build similar abstraction layers.
ReplyDeleteFrankly speaking, that's one of best features of Aexey Belov and Alexey Filatov characters. They do exactly what they need, and nothing more. That's what agile really implies.
> By the way, 25% of schedule is better than average in the industry.
1) I don't like this way of thinking :) I.e. we must be better then average in the industry.
2) The fact is that we finally lost some money - and partially, because of this.
LOL, just found one more nice comment there:
ReplyDelete"I once had an employer who wanted to be able to switch out .NET with Java on the main application for the company. You know, just in case we had an anti-Microsoft client."
I generally agree. I do not defent useless abstractions.
ReplyDeleteIn fact, I fight with them all the time. :)
Hello Alex,
ReplyDeletei think this abstraction discussion is in fact the inability to commit to an particular ORM. ORM is a strategical decision like the programming language, operating system and so on. DO4 provides a high abstraction level from the data storage. That is enough from the point of view of flexibility.
One thing that the "big brains" are ignoring: If you have an abstraction layer and you are really willing to switch the underlying systems, you must test this from the first minute you start your development. Look at the :NET / Mono thing. If you don't test your code on both platforms, it wont run or perform well. In real scenarios this is very time-consuming and expensive.
It is also commonly ignored that you need experience with the underling systems. I'm currently working with Postgre SQL and I'm alone with it. My workmates haven't any experience with it. If you want to be able to operate two different SQL Server systems, you double the training effort.
Absolutely agree: we have nearly the same experience:
ReplyDeleteIf you don't start from supporting several backends of any kind right from scratch (in particular, in tests), the amount of problems you'll get trying to add support for the second one (even if you thought you were well-prepared for this) will be comparable to efforts you need in case there are no any preparations at all.
So IMHO you either should start from supporting them and continuously test the compatibility with each one, if this is really necessary, or simply don't worry too much about this, especially in case the chance you'll really need N-th backend is tiny. Simply because it's a tremendous waste of resources.
And you're absolutely right about training. There are actually lots of essential differences between nearly any two systems. That's very natural: they're competing with each other, and that's the reason of differentiation by features.
ReplyDeleteE.g. theoretically, SQL standard support must imply automatic support of nearly any DB by ORM. But the practical picture is completely different, and e.g. we have two layers of abstraction to deal with this (different SQL DOM providers handling simple differences and Storage-level providers handling more complex ones, such as availability of CROSS APPLY).
And, in almost any case, you must know each backend nearly perfectly to support it. So training time simply can't be ignored here.