DataObjects.Net in 2013 roadmap

This post will cover our plans for DataObjects.Net in year 2013.

Feature update for DataObjects.Net 4.6

Unlike typical bug-fix-only minor releases. DataObjects.Net 4.6.4 will include
several new features.

Support for ignoring certain tables and columns during upgrade

This is essential feature for using technologies such as SQL Server Replication that require special columns to be present in each table.

Support for external 'recycled' definitions

As you know, it's possible to mark certain types and/or fields as recycled to keep corresponding data (tables, columns) available during upgrade. However, this requires keeping some legacy items in your code. External recycled definition will solve this problem by providing an API to add recycled definitions via UpgradeHandlers.

Support for :memory: data source in SQLite provider

SQLite provides special :memory: data source that could be used to operate against in-memory database. This is extremely useful for instance, for testing purposes as the database disappears once connection is closed. Currently, DataObjects.Net requires separate connections for building domain and regular CRUD operations which makes :memory: data source inapplicable. DataObjects.Net 4.6.4 will include support for this data source, limiting one active session per domain.

DataObjects.Net 4.6.4 is expected to be released in March 2013.

Plans for DataObjects.Net 4.7

This is not final roadmap for DataObjects.Net 4.7. Things might change, but here is our current plans.

New faster tuple implementation

DataObjects.Net internally uses special data structure to store Entity fields called Tuple (not to be confused with .NET Framework 4 tuples). Versions prior to 4.7 used set of generic types to store and access tuple fields. Due to the startup slowness of generics and additional memory consumption we switched to non-generic approach that provides more efficient data storage as well as fast typed access. This feature is already implemented and will be included in DataObjects.Net 4.7

Support for reading entities after session has been disposed

This feature would simplify passing entities in ASP.NET MVC and other similar applications.

Automatic caching of generated SQL queries

LINQ translation might take considerable time. DataObjects.Net provides compiled queries to address this. However sometimes it's desirable to cache translation result of each LINQ query. This would be available as an option. By default generated SQL queries will not be cached.

Query result caching API

This is one of the most wanted DataObjects.Net features. We will provide an API to maintain cache of LINQ query results as well as examples how to integrate DataObjects.Net with memcached and other caching facilities.

Change of a default Transactional aspect application

At the moment all your persistent types (i.e. entities, services derived from SessionBound) are powered with our Transactional aspect. This means they automatically provide transaction and activate session upon calling any of their methods. This adds some overhead and most of the users like to manage transactions manually. Thus we decided to remove automatic application of  Transactional aspect in DataObjects.Net 4.7. For those who will need current behavior it would be possible to enable it again by adding special attribute on each assembly with persistent types.

Improved support for native SQL data types

Currently DataObjects.Net supports persistence of standard .NET primitive types and spatial types of SQL Server. We are going to improve this and add support for spatial types in PostgreSQL as well as XML data types for servers that support them. Also operations on such types would be supported by LINQ translator.

Improved logging API

At the moment DataObjects.Net logging is complex to configure.
We're going to change this and provide new more simple interface for it. Also it would be possible to plug your own log consumers.

Reworked validation framework

We're going to improve our validation framework to make it more easy to use and extend with your own validators.

Support for SQL Server Compact 4

SQL Server Compact 4.0 will supported in addition to 3.5

DataObjects 4.7 is scheduled for release in September 2013.
 

Product lifetime policy update

There was no clear policy on DataObjects.Net product lifetime in the past. Now we define such policy. Each major DataObjects.Net release is supported for 18 months. Within this period we will accept bug reports and make bug-fix releases. After product lifetime expires you might still use it "as is". Certain versions might have this period expanded with proper announcement.

Currently we support 3 major DataObjects.Net releases: 4.4, 4.5 and 4.6.

The following table shows when they would go out of support:

• 4.4    April 2013
• 4.5    October 2013
• 4.6    April 2014

As a special exception we extended 4.4 lifetime to make it easy to migrate to newer versions.
The last 4.4 release will be available in April 2013.

DataObjects.Net 4.5.7 and 4.6.3 are released

This is a huge bug-fix release. Everybody is encouraged to upgrade.

DataObjects.Net 4.5.7 and 4.6.3:

• Fixed translation of GroupBy() over columns with same name
• Improved diagnostic message when more than one upgrade handler is provided
• Fixed error after changing case of a persistent field
• Wrap exceptions occured within user code with TargetInvocationException
• Fixed handling of validation aspects on generic types
• Fixed translation of In/Contains over field obtained with `as` operator
• Fixed translation of In/Contains over arbitrary expressions
• Fixed projection to DTO with inheritance
• Fixed translation of DateTime properties access on certain expressions
• Fixed accessing of IGrouping.Key in subqueries
• Fixed implementation of "truncate" operation on DateTime expressions (SQL Server)

DataObjects.Net 4.6.3 only:

• Fixed incorrect column order in queries after certain schema upgrades
• Added DomainConfiguration.NativeLibraryCacheFolder setting
• Significantly improved initialization time for Xtensive.Tuples infrastructure

DataObjects.Net Extensions 4.6.3 and 4.5.7:

Bulk operations extension

•  Remove extra command execution under certain scenarios

Reprocessing extension

• Improve compatibility with DisconnectedState
• Don't use nullable types in ReprocessAttribute properties

DataObjects.Net Extensions 4.6.3 only:

Localization extension

• Improved examples in readme.txt

Tracking extension

• Fixed incorrect handling of partially loaded entities

DataObjects.Net LinqPad driver 4.6:

• Support for DataObjects.Net 4.6 is added

Download

Just as always releases are available at our site and NuGet gallery.

DataObjects.Net 4.5 and 4.6 are updated

Bug fixing updates for DataObjects.Net 4.5 and 4.6 are available.

Here is the list of changes for both versions:

• Fix translation of predicates similar to (object) NullableBool == (object) null
• Fix translation of Union() over boolean columns under certain conditions
• Don't enforce implicit non-nullable/length constraints in inconsistent sessions
• Import DataObjects.Net.targets conditionally when using NuGet package
• Make UpgradeHandler.IsFieldAvailable virtual
• Treat SQL Server errors 3966 and 3971 as serialization failure

The following fixes apply to 4.6.1 only:

• Fix loading/storing of the domain model when multi-database mode is enabled
• Fix assignment of MappingSchema/MappingDatabase for persistent interfaces
• Improve exception message for missing default schema in domain configuration

Just as always you can get new releases at our download site or NuGet gallery.

DataObjects.Net extensions are not updated yet. You could use previous versions of them which are fully compatible with this release.

DataObjects.Net Extensions update

DataObjects.Net Extensions for DataObjects.Net 4.5.5 and 4.6.0 are available.

Changes

•     Added missing xmldocs for BulkOperations extension
•     New Tracking extension (DataObjects.Net Extensions 4.6.0 only)

Tracking extension

Tracking extension provides tracking/auditing functionality on Session/Domain level.

To use it:

1. Add reference to Xtensive.Orm.Tracking assembly

2. Include types from Xtensive.Orm.Tracking assembly into the domain:

<Xtensive.Orm>
  <domains>
    <domain ... >
      <types>
        <add assembly="your assembly"/>
        <add assembly="Xtensive.Orm.Tracking"/>
      </types>
    </domain>
  </domains>
</Xtensive.Orm>

3. To track changes on Session level obtain an instance of ISessionTrackingMonitor through Session.Services.Get<ISessionTrackingMonitor>() method To track changes on Domain level (from all sessions) obtain an instance of IDomainTrackingMonitor through Domain.Services.Get<IDomainTrackingMonitor>() method

4. Subscribe to TrackingCompleted event. After each tracked transaction is committed you receive the TrackingCompletedEventArgs object.

5. TrackingCompletedEventArgs.Changes contains a collection of ITrackingItem objects, each of them represents a set of changes that occured to an Entity within the transaction committed.

Example

1. Subscribe to ISessionTrackingMonitor/IDomainTrackingMonitor TrackingCompleted event

var monitor = Domain.Services.Get<IDomainTrackingMonitor>();
monitor.TrackingCompleted += TrackingCompletedListener;

2. Do some changes to persistent entities

using (var session = Domain.OpenSession()) {
  using (var t = session.OpenTransaction()) {
    var e = new MyEntity(session);
    e.Text = "some text";
    t.Complete();
  }
}

3. Handle TrackingCompleted event call and do whatever you want with tracked changes.

private void TrackingCompleted(object sender, TrackingCompletedEventArgs e)
{
  foreach (var change in e.Changes) {
    Console.WriteLine(change.Key);
    Console.WriteLine(change.State);

    foreach (var value in change.ChangedValues) {
      Console.WriteLine(value.Field.Name);
      Console.WriteLine(value.OriginalValue);

      Console.WriteLine(value.NewValue);
    }
  }
}

Sync extension

Sync extension was planned for 4.6 release. We decided that it is not production ready and postponed it until 4.7.

Download

• From our site.
• From NuGet gallery.

Advanced mapping in DataObjects.Net 4.6 — Part 1

This article starts a series of posts covering advanced mapping capabilities introduced in DataObjects.Net 4.6.

Idea

The idea of advanced mappings is simple — to have a way to spread groups of persistent classes among different database schemas and even different databases according to number of mapping rules.

Basics

Advanced mapping configuration is based on set of mapping rules, each of them defines how to map a particular entity. Mapping rule consists of two logical parts:

• condition (defines when the rule is applied)
• mapping (specifies in what database/schema to place the target)

The rules could be specified in configuration file or right in code using fluent API. They are processed in the order they appear, so the first matching rule wins.

In addition to collection of rules, two properties specify default mapping: DefaultSchema and DefaultDatabase. These properties are used as fallback values if no rule matches entity being processed. Note, that 'defaultSchema' option is mandatory if any rules for schemas are specified.

XML configuration

OK, we are done with theory. Let's write some real mapping configurations. The most trivial one is setting the default schema. This is already supported by DataObjects.Net since version 4.3. Let's recall it:

<domain defaultSchema="myapp" ... />

Starting from version 4.6, we can write more precise mapping rules. They are defined in <mappingRules> element inside domain configuration.

<domain defaultSchema="myapp" ... >
  <mappingRules>
    <rule namespace="MyApp.Model.Foo" schema="myapp_foo" />
  </mappingRules>
</domain>

The above example maps all types from namespace 'MyApp.Model.Foo' to schema 'myapp_foo'. All other types are mapped to schema 'myapp' (fallback rule). If you have model with multiple assemblies in might be convenient to map to assembly names instead of namespaces:

<domain defaultSchema="mysite" ... >
  <mappingRules>
    <rule assembly="MySite.Model.Blog" schema="mysite_blog" />
    <rule assembly="MySite.Model.Forum" schema="mysite_forum" />
  </mappingRules>
</domain>

The above example splits model into two schemas. One for persistent types related to blog module. The other one is for forum module. The rest is mapped to 'mysite' schema. 

Fluent configuration

As alternative to XML-based configuration you can programmatically define mapping in code. Mapping rules could be added by using DomainConfiguration.MappingRules collection. Two examples from previous section will look like this:

domainConfiguration.DefaultSchema = "myapp";
var rules = domainConfiguration.MappingRules;
rules.Map("MyApp.Model.Foo").ToSchema("myapp_foo");

domainConfiguration.DefaultSchema = "mysite";
var rules = domainConfiguration.MappingRules;
blogAssembly = typeof (MySite.Model.Blog.Post).Assembly;
forumAssembly = typeof (MySite.Model.Forum.Thread).Assembly;
rules.Map(blogAssembly).ToSchema("mysite_blog");
rules.Map(forumAssembly).ToSchema("mysite_forum");

The last example uses hypothetical 'MySite.Model.Blog.Post' and 'MySite.Model.Forum.Thread' types to discover the corresponding assemblies.

To be continiued

This post covered basic ideas as well as mapping to multiple schemas. The second part will cover mapping to multiple databases and related application design techniques. Stay tuned.

Issues with recent PostSharp

Recent versions of PostSharp (2.1.7.17 and 2.1.7.18) does not work well with DataObjects.Net. If you installed DataObjects.Net from NuGet and get the following error when compiling:

Error in MulticastAttributeUsage applied on type 'Xtensive.Orm.PersistentAspect': attribute inheritance is not supported on properties and events.

you should downgrade your PostSharp to version 2.1.7.16.

To do this open package manager console (Tools ->Library package manager -> Package Manager Console) and execute the following commands:

> Uninstall-Package PostSharp -Force
> Install-Package PostSharp -Version 2.1.7.16

Recently released DataObjects.Net 4.5.5 and 4.6.0 explicitly set maximal supported version of PostSharp to 2.1.7.16 so you will not get this error when using them.

DataObjects.Net 4.6 goes final

I'm glad to announce that DataObjects.Net 4.6.0 is released. This post provides an overview of changes and improvements in 4.6 version. They would be explained with more details in future posts.

SQLite provider

SQLite has joined the list of supported databases.

Quick domain configuration example:

    <domain name="mySqliteDomain"
            providerName="sqlite"
            connectionString="Data Source=mydatabase.db" />

You'll also need to add reference to Xtensive.Orm.Sqlite.dll assembly in your project.

Mapping to multiple schemas and even databases

Since 4.6 it's possible to map persistent model to multiple schemas and databases.

The following examples maps model to two schemas based on entity namespaces:

    <domain name="myDomainWithTwoSchemas" defaultSchema="dbo">
        <mappingRules>
            <rule namespace="MyApp.Model.Foo" schema="foo" />
            <rule namespace="MyApp.Model.Bar" schema="bar" />
        </mappingRules>
    </domain>

I'll cover multi-mapping configurations in future posts.

Upgrade improvements

In this version we heavily reworked upgrade to be more fast and reliable. First major improvement is extensive usage of parallel computations, so now DataObjects.Net will perform some actions in parallel to build domain faster. Second major improvement is wrapping all upgrade actions in a single transaction if database supports transactional DDL. Previous versions of DataObjects.Net used two transactions in Perform and PerformSafely modes because additional upgrade domain was created.

Simplified key generator API

In DataObjects.Net 4.6 we changed API for key generators (both standard and user-written) to be more simple and intuitive. For example for creating custom key generator table or sequence you no longer need to write domain service. Simple [KeyGenerator(Name = "Foo")] will work just well. I'll explain new key generator API in details in future posts.

Final words

Just as always you can download packages from our site or NuGet gallery.

DataObjects.Net 4.5.5 released

DataObjects.Net 4.5.5 is made available. This is bug-fixing release for 4.5 branch. List of resolved problems could be found here. Just as always you can download packages from our site or NuGet gallery.

DataObjects.Net 4.5.5 RC and 4.6.0 RC

Two release candidates for 4.5 and 4.6 branches are made available.

Here is short list of changes:

DataObjects.Net 4.6.0 RC

Added NamingRules.RemoveDots and NamingRules.RemoveHyphens

New options work similar to existing UnderscoreDots and UnderscoreHyphens except that dots and hyphens are completely removed from table or column name instead of replacing with underscore.

Added Connection and Transaction properties to UpgradeContext

DataObjects.Net provides single transaction for upgrade process. Those properties allows to execute custom actions at any stage.

Added Session property to UpgradeContext

This is new recommended approach for getting current session in upgrade handlers.

UpgradeStage.Initializing is marked obsolete

This stage no longer occurs in DataObjects.Net 4.6 so corresponding warning would be emitted when it is used.

Added OnPrepare and OnComplete methods to UpgradeHandler which are executed before and after any upgrade

OnPrepare partially replaces old Initializing stage. It is executed are very early stage so you can apply fixes to the database schema before any upgrade actions are performed.
OnComplete is executed at the very last stage and could be used for configuring domain before it is returned from Domain.Build() method.

Added advanced version of domain builder module

New method OnAutoGenericBuilt() allows to change created automatic generic instances.

Correctly clean up key generator tables if entities are created in OnUpgrade() method

If entities are created in OnUpgrade() method key generator tables (i.e. Int64-Generator) might be not empty after the upgrade. This is fixed now.

Several methods that implicitly use Session are marked as obsolete

Since DataObjects.Net 4.4 we are recommending explicitly pass session everywhere.
Some old methods that implicilty use Session.Current has been marked as obsolete.
Examples of such methods InvokeTransactionally() extension method overloads that don't take session and GetTypeInfo() extenstion method.

DataObjects.Net 4.5.5 RC

Omit redundant parentheses in generated SQL for chains of set operations

This allows to write rather large chains of .Union() calls.
See this bug report for details.

Correctly handle hierarchy split when doing upgrade

This fixes error about TypeId removals when hierarchy has been split to hierarchies with single type.

Fixed translation of Contains/Any/All under certain scenarios

See this bug report for details.

Correctly parse pre-release versions of MySQL (e.g. 5.5.25a)

See this bug report for details.

Other changes

Debug symbols (.pdb) are available for this and all future release. They require source code license
just like source code package.

Final words

Just as always distribution packages are available on our download site and NuGet gallery.