News, examples, tips, ideas and plans.
Thoughts around ORM, .NET and SQL databases.

Friday, January 14, 2011

How to handle non-persistent collection

Today we've faced with an interesting scenario from Jensen: an Entity has a non-persistent collection which should be serialized to and deserialized from a persistent field on demand. Here is how the problem was initially described on the support website (go to the end of the page):

Object A should contain a list of samples (object T). I could do something like
A a = GetA();
a.Load(); (convert the byte[] to a List)
and something like
a.Store(); (convert the List to a byte[])
m_transactionScope.Commit();
But I don't like calling a Load() and Store() function each time, it would be easier if this could be automated. When A is loaded from the database it should Load(), when saved to the database it should Store().

After several experiments I've found a solution.

1. Add a set of properties to the desired class.
public class MyEntity : Entity {
    ...
    [Field] // Is used as an indicator that collection has been changed
    private bool IsChanged { get; set; }

    [Field] // Backing field for serialized collection value
    private byte[] CollectionValue { get; set; }

    // The collection itself. Note that it is not persistent.
    public ObservableCollection<int> Collection { get; set; }
    ...

2. Add proper methods for serialization & deserialization:
    private static byte[] Serialize(ObservableCollection<int> collection)
    {
      var ms = new MemoryStream();
      var bf = new BinaryFormatter();
      bf.Serialize(ms, collection);
      return ms.ToArray();
    }

    private static ObservableCollection<int> Deserialize(byte[] bytes)
    {
      var bf = new BinaryFormatter();
      var ms = new MemoryStream(bytes);
      return (ObservableCollection<int>) bf.Deserialize(ms);
    }

3. Add Entity initializer & event handlers:
    protected override void OnInitialize()
    {
      base.OnInitialize();
      // Initializing the collection
      if (CollectionValue != null && CollectionValue.Length > 0)
        Collection = Deserialize(CollectionValue);
      else
        Collection = new ObservableCollection<int>();

      // Subscribing to events
      Collection.CollectionChanged += Collection_CollectionChanged;
      Session.Events.Persisting += Session_Persisting;
    }

    void Session_Persisting(object sender, EventArgs e)
    {
      if (!IsChanged)
        return;

      // Serializing the collection right before persist, if it is changed
      CollectionValue = Serialize(Collection);
      IsChanged = false;
    }

    void Collection_CollectionChanged(object sender, NotifyCollectionChangedEventArgs e)
    {
      IsChanged = true;
    }

So, what is going on here?
An instance of MyEntity is subscribed to a CollectionChanged event of MyEntity.Collection and updates MyEntity.IsChanged property when CollectionChanged event is fired. As the property is persistent, the MyEntity instance is added into a queue for persisting. Right before the persist procedure takes place, the instance of MyEntity receives a notification from its Session. During handling of this event we check whether the collection has been changed or not. If so, we update MyEntity.CollectionValue property and changes are successfully persisted to a database.

Note, that I've decided to use ObservableCollection type instead of List because the solution requires a notifications from the collection.

Hope this helps.

Thursday, January 13, 2011

Making connection strings secure

Recently, Paul Sinnema from Diartis AG asked on the support website how to make connection strings, listed in application configuration files (app.config or web.config), more secure.


For that time the only thing we could suggest was to use a workaround like that:

1. Encrypt the required connection string with encryption method you prefer and put it somewhere in web.config/app.config file in encrypted form.

2. Load DomainConfiguration through standard API:

var config = DomainConfiguration.Load("mydomain");

3. Set manually config.ConnectionInfo property with decrypted connection string

config.ConnectionInfo = new ConnectionInfo("sqlserver",
  DecryptMyConnectionString(encryptedConnectionString));

4. Build Domain with the config.

Paul answered that although the workaround is useful, he'd prefer that DataObjects.Net could be integrated somehow with standard .NET configuration subsystem that provides easy encrypting/decrypting of connection strings.

Let me explain the approach, Paul was talking about. For example, if plain connection string configuration looks like that;

  <connectionStrings>
     <add name="cn1" connectionString="Data Source=localhost\SQL2008;Initial Catalog=DO40-Tests;Integrated Security=True;MultipleActiveResultSets=True" />
  </connectionStrings>

then after executing "aspnet_regiis.exe -pef "connectionStrings" %PATH_TO_PROJECT%" the encrypted one will look like the following:

  <connectionStrings configProtectionProvider="RsaProtectedConfigurationProvider">
    <EncryptedData Type="http://www.w3.org/2001/04/xmlenc#Element"
      xmlns="http://www.w3.org/2001/04/xmlenc#">
      <EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#tripledes-cbc" />
      <KeyInfo xmlns="http://www.w3.org/2000/09/xmldsig#">
        <EncryptedKey xmlns="http://www.w3.org/2001/04/xmlenc#">
          <EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#rsa-1_5" />
          <KeyInfo xmlns="http://www.w3.org/2000/09/xmldsig#">
            <KeyName>Rsa Key</KeyName>
          </KeyInfo>
          <CipherData>
            <CipherValue>TJETPde0GVrSAKk0VpUs3EsV8XFRQrVU1lGlrX8aEhvfa4MaMpN3O6KLo3I9e5oujXBImKtmXHWxnF6KRcSWteuQXD4I37Dq6yE1cZ4qDHqNMkcQK+lgi6jbRLcPzYQdFuOLagsAVpif8OvY0dJkiKd+Srqjwz7ZO225VCipaQs=</CipherValue>
          </CipherData>
        </EncryptedKey>
      </KeyInfo>
      <CipherData>
        <CipherValue>6bQyCUdH9rqdNAD4IwmSXtAQIEuRfSjpQ/Py5ovz2ZpwJdEXa4W3OKWMm52g1ARXPqMYw1thfQae//+1JWTAPQwrE3JTZEdcgu47KE2x8uqdWeJfoXSecp6berSPc4qAa4F2B5x8weUGPiAsBldVfBXwUBPniKWbnnXViJWmcSn16kaAtIkq5GtNOp7YhPt9WKZG0QkFRpEaRAWqLgxK6nXUwK8BgbGEBI4B27QSFj2ME1/JI+o//RrchntEuW6VolEA7PAejMujsIn1xb0hqg==</CipherValue>
      </CipherData>
    </EncryptedData>
  </connectionStrings>

As a result, the connection strings get encrypted and therefore, much more secure. Working with them is as simple as usual: they can be accesses through standard ConfigurationManager without any special tricks:

ConfigurationManager.ConnectionStrings["cn1"].ConnectionString
And this simplicity makes the whole approach extremely useful.

So, what could be done to get DataObjects.Net compatible with all this shiny stuff, keeping in mind that connection strings section is outside DataObjects.Net configuration?

After thinking for a while on the problem, we've got the following idea: what if we put in domain configuration not a connection string itself, but a reference to the connection string, that is actually defined in connectionStrings section? For instance:

<Xtensive.Orm>
    <domains>
      <domain provider="sqlserver" connectionString="#cn1" ... />
    </domain>
  </domains>
<Xtensive.Orm>
Where:
  • cn1 is a name of a connection string defined in connectionStrings section;
  • # sign is used to inform that actually this is a reference to a connection string and DataObjects.Net must resolve it through ConfigurationManager.
Having this feature implemented, DataObjects.Net will provide support for encrypted connection strings out-of-the-box.

And now, we'd like to ask you some important questions:
  • What would you say about the feature?
  • Will it be useful for you and should it be implemented?
  • Is the "#name" notation for setting a reference to a connection string is more or less clear or wouldn't it better to change it to something else?

Dear users of DataObjects.Net, we need your feedback. Please, post a comment or two.

Thanks in advance.

Wednesday, December 15, 2010

Christmas Sale



We are glad to announce that Xtensive.com Christmas Sale has started today, December 15 and will last till the end of the year. During this time you get 20% off any edition of DataObjects.Net and Help Server. To apply this 20% Christmas special to your order, just enter the following coupon code at the shopping cart or product checkout page: DEC2010PROMO

Should you have any questions, please email to sales@x-tensive.com

Merry Christmas and Happy New Year!

Sincerely,
DataObjects.Net Team

Thursday, December 09, 2010

DataObjects.Net v.4.4 Beta, v.4.3.6 and the Christmas Sale


Development branch

Today we are happy to announce the next significant step in DataObjects.Net 4 development — DataObjects.Net 4.4 Beta 1. This release includes the following exciting features and updates:
  • ORM platform unification for usage in both server-side and client-side applications
  • Optional Session auto-activation
  • Optional Transactions
  • 'Unit of work' pattern support in UI applications
  • Greatly reduced number of assemblies (only 3 assemblies instead of 17)
  • F# language support
  • Visual Basic language support
  • Xtensive Practices initiative
  • and much more...
For further information see DataObjects.Net 4.4 Beta 1 Release Notes.

Due to major changes in API, usage of this release in projects, based on the previous version of DataObjects.Net, might require some fixing of assembly references or namespaces. Migration path is described in the Release Notes file.

Download DataObjects.Net 4.4 Beta 1

Stable branch

DataObjects.Net 4.3 branch is updated to 4.3.6 version. This release includes the following changes:
Features
  • Visual Basic support
Updates
  • EntitySet now implements IValidationAware
  • SessionOptions.ReadRemovedObjects is added
Bug fixes
Download DataObjects.Net 4.3.6

And one more thing

Xtensive LLC announces the Christmas Sale, which is starting in the middle of December.
Look for Christmas coupons on our websites, take your chance and get 20% off regular product price.

Stay tuned.

Monday, November 08, 2010

DataObjects.Net vs NHibernate: conceptual differences and feature-based comparison

I written the document that might help the people choosing among DataObjects.Net and NHibernate to make a decision. Currently its second part related to feature-based comparison (feature matrix) is actually a preliminary one (that's explained in the document), so later I'll rewrite it. But anyway, I hope this will be helpful.

As usual, any comments and suggestions are welcome.

Thursday, November 04, 2010

New DataObjects.Net website

... has been launched!

P.S.
In case you find anything that doesn't work as expected, please, get us informed.

Thanks in advance.

Monday, November 01, 2010

Development news

Sorry for keeping the silence - I really haven't been writing here for a while. Usually this means we're quite busy with some new stuff, and in this case this is absolutely true. We're working on two main features we're going to bring quite soon:
  • DataObjects.Net v4.4 - it brings really dramatic changes; most important of them are described in the following part of this post.
  • New, dedicated DataObjects.Net web site. We know we should do this much earlier, and now even our own patience is finished ;)
Some details:

Website

It is almost finished, and will be launched quite soon. Few screenshots:

 
 

Hopefully, you'll like it.

DataObjects.Net v4.4

As I wrote, we implement a set of dramatic changes there, but mostly they are related to refactoring we planned for pretty long time. Most important changes there include:
  • Elimination of most of of assemblies. We'll leave just Xtensive.Core, Xtensive.Aspects, Xtensive.Aspects.Weaver (not necessary to reference it) and Xtensive.Orm, so you should reference just 3 assemblies from your code. It's a result of significant internal refactoring + ILMerge.
  • We're starting to use actual assembly version numbers from this release. All assemblies there are marked as v4.4.0.0 (their file versions reflect minor revisions and build numbers). Assembly versions will be changed on each significant API change. Project upgrade tool will help to automatically upgrade your projects to the new version.
  • Namespace renaming:
    Xtensive.Storage is split into Xtensive.Orm (most of classes you need are there now) + Xtensive.Storage;
    Xtensive.Core.* are renamed to Xtensive.* (but Xtensive.Core itself is left as is);
    Xtensive.Modelling become Xtensive.Core.Modelling;
    Xtensive.Integrity is gone;
    there are few other less important changes.
  • Usage of Session.Current / Session.Demand() is optional now. This actually means explicit passing of Session is recommended now; old API fully works, but all such methods are marked as obsolete; there are new methods for any part of API that formerly relied on current Session (e.g. there is Session.Query.All<T>()). Automatic session activation on method call is optional now (this behavior is managed by one of SessionOptions flags); DO itself does not require it, but it can be used when it is convenient (e.g. this is still a recommended practice for web applications). We did this because of two issues:
    a) We discovered people tend to think DO isn't designed for concurrent operations (such as passing persistent objects to another thread). Of course, this isn't true, but "current Session" concept makes a different impression. There were people stating it as one of blocking issues.
    b) People used to different style, and finally we understood it's a bad idea to make them to study our own one.
    c) The advantage of this style is really subjective. It hides the complexity, although people used to manage it by their own in this case. Moreover, it doesn't bring any essential advantages to us - i.e. actually, framework does not need this feature to be enough extendable and flexible.
    d) This technique isn't friendly to new "async" and "await" in C# 5 (i.e. there shouldn't be async methods in SessionBound descendants), so its usage must be 100% optional.
  • SessionOptions brings 3 pre-defined combinations of flags now: ClientProfile, ServerProfile and LegacyProfile. Legacy profile is the mode you use now. Server profile is legacy profile without automatic session activation (since now this is optional). And finally, there is a client profile: each Session operating in this mode is created along with attached DisconnectedState, so it works nearly as DataContext in EF or Session in NH (from many points - even better). This mode allows people using different ORM frameworks to migrate to DO a bit easier.
  • Xtensive.Practices is finally separated there. Btw, it will be shipped with full source code.
All above changes are mainly related to API. But certainly, this isn't all:
  • We've actualized Manual. Btw, it's split in multiple topics in help, so it's much easier to study it there.
  • Samples are now organized by technology. Also, they don't require IIS and installation of "IIS Metabase and IIS 6 configuration compatibility" components for Windows to be executed from VS.NET. Sandbox projects are moved to samples. Finally, there is a good Readme.htm that fully explains how to create a sample database, compile and and run them.
  • We'll ship a set of .RAR archives allowing to download only necessary part of each build (e.g. just binaries). Actually, we already are.
All these changes (along with the dedicated web site) must make the product much more user-friendly - especially, for its adopters.

v4.4 will be released during November.

Further plans

We're pushing the development of DataObjects.Net in two directions:
  • Adding new and unique features and APIs. That's what distinguishes us from others. Frankly speaking, we've paused our activity here for a certain period, so our near-time goal is to push this part forward.
  • Adding old and widely adopted features / APIs. That's what makes DO easier to study and more compatible with others. We must propose an alternative to nearly any nice feature available in competing products. 
As you see, initial release of v4.4 is mostly focused on initial user experience and compatibility / similarity with competitors, but its subsequent updates (v4.4.X) will be devoted mainly to extensions.

Our further (pretty near) plans include:
  • Implementation of a small framework simplifying MVVM development with DO. The description is here (a bit outdated).
  • Implementation of access control / security extension - the preliminary description is here.
P.S. Right now you can download new DataObjects.Net v4.3.5 build 6600.

DataObjects.Net v4.3.5 is updated to build 6600

This update includes a set of bugfixes; probably, the most important one is multithreading bug in prefetcher.

IMPORTANT, PLEASE READ CAREFULLY:

1. You must upgrade to this version, if your trial is expired today. A hard-coded limitation of operational period is extended to February 1st, 2011 in this version. Certainly, this is related to trial version only. 60-day expiration of trial version is still intact.

2. It's strongly recommended to change just DataObject.Net assemblies while upgrading to this version. You must run the upgrade in PerformSafely or Perform mode.

3. We've fixed issue 813 in this version; the issue affects on default mapping. So if you used:
  • [TableMapping] attribute on any of tables with EntitySets describing many-to-many relationships with automatically mapped intermediate item (e.g. Book.Authors and Author.Books),
  • or used [FieldMapping] on such EntitySet fields (this didn't work earlier, but now this works),
you must:
  • Back up your database before upgrade to this version. Count the number of tables there.
  • Manually rename the tables where intermediate items for such EntitySets are stored accordingly with new mapping rules. Old table name is "OwnerTypeName-PropertyName-ItemTypeName", new table name is "OwnerTableName-PropertyMappingName-ItemTableName".
  • Upgrade the Domain and count the number of tables there. If it's the same, and tables of intermediate items aren't empty, the upgrade was successful.
The new installers can be found in Downloads section.

Wednesday, October 06, 2010

DataObjects.Net v4.3.5 is updated to build 6299

This update includes a set of fixes related to:
  • LINQ (several issues)
  • INotifyCollectionChanged implementation in EntitySet - we discovered there were few WPF-specific issues, e.g. in some cases WPF tries to enumerate an EntitySet of already dead (removed) entity.
  • Order of events in paired property sync scenario (earlier I described this scenario).
  • Proper assignment of command timeout - in particular, during schema upgrade.
Sorry, no links to issue pages. All these issue are minor (some of them were fixed without crating an issue in public tracker at all), so ether you already know about some of them, or they aren't important for you ;)

Tuesday, October 05, 2010

Tip: tuning up Google Chrome / Firefox to search @ our help and support web sites faster

Both Google Chrome and Mozilla Firefox support search shortcuts (or custom search providers): if you prefix an URL you enter by one of keywords, the browser transforms the text behind that keyword to URL by a customizable rule, and opens it.

I'm going to show how to tune up Chrome to add a set of useful search shortcuts for help.x-tensive.com and support.x-tensive.com. Steps for Mozilla Firefox must be very similar.

Open "Options" page in Chrome, and click "Manage" button at "Default search" section there:


You must see nearly the following page. I already configured 3 custom search providers in my browser:


To add these 3 providers, click on "Add" button and fill the form as follows:


Repeating this once more:


And finally (full URL here is "http://support.x-tensive.com/search/?q=%s&t=question"):


As result, you'll be able to search through our help and support web sites faster.

Type "xh " in address bar:


Then, type a keyword to find in help index:


And hit "Enter". The result:


Adding similar search shortcuts for StackOverflow and MSDN is a good idea as well :)