DataObjects.Net Contribution Program Progress

Hi All,

As you might remember, in February Xtensive LLC officially announced the DataObjects.Net Contribution Program. Frankly speaking, we didn't even imagine what impressive results it would bring.

First of all, I have to mention Csaba Beer and its efficient work on Firebird provider. Csaba developed Sql.Dom driver and now the entire Firebird provider is already integrated into DataObjects.Net ORM core (branch 4.5). It is being tested, fixed and polished by Xtensive LLC engineers. The alpha version will be available soon, may be even this week.

Secondly, Malisa Ncube joined the Contribution Program and chose MySql provider as the goal. After several weeks of hard work the main part of Sql.Dom driver is almost finished. Today I've even managed to run all ORM tests on MySql. Unfortunately, not all of them finished successfully but that's OK for the first time. The work on the provider might require a week or two and after that it will be ready for the alpha.

Last but not least, Entity Model Designer for DataObjects.Net that is being actively developed by Peter Šulek, appeared like a miracle for most of us. As you know, DataObjects.Net always was strictly code-centric tool and there were no designers or any other visual tools for it. But with the help of Peter the things have been principally changed to better side. While the tool itself is in alpha phase, it looks very promising. Peter promised to blog about the designer soon, so let's wait for the overview or tutorial.

All of these brave and bright developers will receive DataObjects.Net Ultimate editions as a reward for their outstanding efforts.

Wanna see yourself in the list? Choose a feature or two from the feature list and join da game! The reward is guaranteed.

Nightly builds are back

Hello All,

After some intense efforts of our developers the nightly builds are back and starting from this weekend are available on the downloads page. The first one (build #7299) is already published.

Next builds will appear in the section on a regular basis as soon as fixes are applied to the codebase. The current technological process provides the ability to produce a fresh build within less an hour, so I am sure you will appreciate the innovation.

There is still a couple of tasks that should be done to achieve better level of usability and transparency: for instance, a build should contain information about the changes that are made in it, so a visitor could get an idea which one he should download and install.

Another exciting bit of news concerning nightly builds is that they will be available for everyone, not only for owners of Professional & Ultimate editions as it is stated on the prices page. The page will be updated soon to reflect this change.

Thank all of you who use DataObjects.Net, who visit our support website, who report tricky bugs and request new awesome features. We are trying our best to deliver you mature and solid framework. Thanks for your choice.

On security system, part 1

This is an introductory post to the security system design and implementation in DataObjects.Net. Here we'd wanted to define common terms, considerations and requirements to the upcoming security system.

Bits of theory

Almost any access control model can be stated formally using the notions of users (subjects), objects, operations, and permissions, and the relationships between these entities.

• The term user refers to people who interface with the computer system directly or not and on behalf of whom some actions are being taken by a computer program or a process.
  
• An object in terms of classic OR/M can be any entity or a group of entities accessible within the mapped database(s).
  
• An operation is a standalone action invoked by the user on the objects.
  
• Permissions (or privileges) are authorizations to perform some action on
  the objects. The term permission refers to some combination of object and operation.
  

The role-based access control model (RBAC) adds one more fundamental term to the list — a role. A role is essentially a collection of permissions. Within an organization, roles are relatively stable, while users and permissions are both numerous and may change rapidly. Controlling all access through roles simplifies the management and review of access controls, therefore we'd prefer to follow role-based security model where users receive permissions only through the roles to which they are assigned.

Another advantage of the role-based access control model is the fact that roles are initially hierarchical — roles can inherit permissions from other roles. As a result, appropriate role hierarchies can be flexibly defined for any business process workflow, for example:

As a conclusion: although any access control system has its own advantages and limitations, we've chosen the RBAC one as a base for access control model in DataObjects.Net because of its flexibility and efficiency in the most usage scenarios.

Requirements and other considerations

First of all, we don't want to reinvent the wheel (again). If any core part of the standard .NET security system can be consumed, then it should be consumed. Mainly, I imply core interfaces such as IPrincipal, IIdentity, etc. This might help to use Thread.CurrentThread.Principal property in the same way as we use Thread.CurrentThread.CurrentCulture in localization extension, as well as more tightly integrate with system authentication services.

Other considerations:

• Security-related data mustn't be stored in serialized way in blob fields or something. It must be accessible via plain SQL.
  
• If this is possible, security system should be implemented as as extension (separate assembly) to the core framework.
  
• Security policy shouldn't be automatically applied to all persistent types. Only selectively chosen and configured persistent types should be subject for security system. This could be done with the help of special interface marker, attribute usage or similar.
  
• Authentication part should be extensible with custom types of authentication services (LDAP, WebServices, etc.).
  
• LINQ queries should be transparently re-written by security system to apply effective permissions.
  
• ASP.NET membership provider should be implemented as well.
  

This list doesn't pretend to be complete. Something might got out from our sight. If so, please don't hesitate to post a comment.

In the next posts of the series I'll try describing several aspects of the system in more detailed manner.

DataObjects.Net Contribution Program

Hello All,

Starting from the version 4.4 we are happy to officially announce the DataObjects.Net Contribution Program. The main goal of the program is to speed up the development process of the ORM by getting involved contributors from outside Xtensive LLC company and award them the eternal glory and of course, licenses.

Frankly speaking, we'd tried such an approach experimentally earlier. As a result, the PostgreSQL data provider was build by Ottó Havasvölgyi, and he gained the Ultimate license for that excellent work.
This winter another talented person, Csaba Beer, is participating in Firebird data provider development. And again, the Ultimate license is what he will definitely get in return.

As for now the scheme works well, we've decided to officially approve the approach and right now we are searching for volunteers to help implementing the following data providers:

• MySQL data provider
  
• SQLite data provider
  
• EffiProz-DB data provider
  

As you might noticed, there were several requests concerning MySQL & EffiProz-DB support. As for the least one, even developers from EffiProz Limited are ready to help with implementation and testing. The Contribution Program is not limited to data providers only, you might be get more interested in any other feature request, that are listed here.

To participate in the Contribution Program, please write to info@dataobjects.net.
Thanks.

DataObjects.Net 4.4 goes final

We are very excited to present the final version of DataObjects.Net 4.4. It took almost 5 months of intensive work for our small but brave team to accomplish this task. We tried to include the most awaited features, simplify the overall API and make DataObjects.Net more friendly for desktop application development.

Contents:
1. Changes comparing with DataObjects.Net 4.4 Beta 2
2. Changes comparing with DataObjects.Net 4.3
3. Download DataObjects.Net 4.4
4. What about the future versions?

Changes comparing with DataObjects.Net 4.4 Beta 2

There are 2 minor changes comparing with Beta 2 version.

Usability of GetProperty/SetProperty
Firstly, the usability of Persistent.GetProperty/SetProperty methods is extended. Both methods now support dot-like notation, for example:

string customerName = orderItem.GetProperty("Order.Customer.Name");

Moreover, both methods support not only publicly visible persistent properties, as it was in earlier versions of DataObjects.Net but all range of them: public, protected, internal and private.

Localization
Secondly, we've reworked the localization sample, updated and extracted common interfaces and types into separate assembly 'Xtensive.Practices.Localization'. This feature provides the ability to define localizable entities, localizable fields, describe backing fields for them and execute LINQ queries absolutely transparently.

Here is the example of localizable entity:

[HierarchyRoot]
public class Page : Entity, ILocalizable<PageLocalization>
{
 [Field, Key]
 public int Id { get; private set; }

 // Localizable field
 public string Title
 {
  get { return Localizations.Current.Title; }
  set { Localizations.Current.Title = value; }
 }

 // Here localizations are stored
 [Field]
 public LocalizationSet<PageLocalization> Localizations { get; private set; }
}

Current localization setting is managed via LocalizationScope or Thread.CurrentThread.CurrentCulture property.

// Editing localizable properties through localization scope
using (new LocalizationScope(new CultureInfo("en-US"))) {
        welcomePage.Title = "Welcome!";
}               

// The same action but through Thread.CurrentThread.CurrentCulture
Thread.CurrentThread.CurrentCulture = new CultureInfo("en-US");
welcomePage.Title = "Welcome!";

The most valuable part is the transparency of LINQ queries, you don't have to add or change anything. Special query preprocessor, located in 'Xtensive.Practices.Localization', is automatically attached to query processing pipeline and does all the work for you.

Thread.CurrentThread.CurrentCulture = new CultureInfo("es-ES");
using (var session = domain.OpenSession()) {
        using (var ts = session.OpenTransaction()) {

                var query = from p in session.Query.All()
                where p.Title=="Bienvenido!"
                select p;
                Assert.AreEqual(1, query.Count());

                ts.Complete();
        }
}

In order to add localization support to DataObjects.Net-based projects, follow the instructions in the corresponding chapter in DataObjects.Net 4.4 Release Notes.

Changes comparing with DataObjects.Net 4.3

Better WPF support
DataObjects.Net 4.4 development branch embodies an idea of ORM platform unification for usage in both server-side and client-side applications. Previous version of DataObjects.Net with thread-bound and automatically activated Sessions as well as transactional properties, methods, etc. fits well in server scenarios, but client application development, such as WPF-based ones, might cause difficulties, especially in multi-threaded environment model and delayed binding of windows controls to Entities.

Keeping this in mind, we decided to alter and extend Session behavior. First of all, we made 'Automatic Session activation' and 'Automatic Transaction opening' modes optional, moreover, they are switched off by default. Secondly, in this version we are introducing Session profile notion which represents a combination of predefined Session options for a particular scenario, for instance, WPF application or ASP.NET website.

For now there are three main profiles: Server profile, Client profile & Legacy profile.

1. Server profile is intended to be used in server scenarios, such as ASP.NET, ASP.NET MVC, WCF Data Services, etc. It doesn't include 'Automatic Session activation' and 'Automatic Transaction opening' modes, so several Session instances could be used within a thread and all actions with Entities, Structures & EntitySets require the presence of opened Transaction. This profile is the default one for DataObjects.Net 4.4.

2. Client profile is designed for WPF, Windows Forms, Console applications, whatever. It uses the new 'Disconnected' Session mode which means that Session encapsulates an instance of DisconnectedState and runs in occasionally connected conditions, logs modifications of Entities and applies them to a database on request. It doesn't include 'Automatic Session activation' either but has 'Automatic Transaction opening' option switched on (in this mode it uses local in-memory transactions, not database ones, in order to log Entity modifications).

3. To preserve compatibility with Session & Transaction behavior from previous version of DataObjects.Net we added special profile named Legacy, which has both 'Automatic Session activation' and 'Automatic Transaction opening' modes switched on.

PostgreSQL 9.0 support
Starting from version 4.4 Beta 2, DataObjects.Net provides support for recently released PostgreSQL 9.0. Note, that the 2.0.11 or higher version of Npgsqlprovider assembly is recommended for usage with this version of PostgreSQL.

Visual Basic support
Visual Basic is fully supported starting from DataObjects.Net 4.4. LINQ translator understand and translate properly the following Visual Basic's specific methods for String type: LTrim, RTrim, Len, Left, Right, Mid, UCase, LCase. Auto properties are supported as well. Here is an example model:

Imports Xtensive.Orm

Namespace Model
  Public Class NonPersistent
    Public Property Id As Integer
  End Class

  
  Public Class Author
    Inherits Entity
  
    Sub New()
      MyBase.New()
    End Sub


    
    Public Property Id As Integer

    
    Public Property Name As String

    
    Public Property Books As EntitySet(Of Book)

  End Class

F Sharp support
DataObjects.Net 4.4 declares FSharp support. Here is a simple model, written in FSharp. Unfortunately, FSharp doesn't support auto properties as C# & VisualBasic do, so you have to write the appropriate code to emulate them.

module Model
open Xtensive.Orm

[]
  type Person() =
  inherit Entity()

  []
  member this.Id
  with get() = this.GetFieldValue "Id"
  and private set(v: int) = this.SetFieldValue ("Id", v)

  []
  member this.Name
  with get() = this.GetFieldValue "Name"
  and set(v: string) = this.SetFieldValue ("Name", v)

Download DataObjects.Net 4.4

The final version is available for download on the official website.

What about the future versions of DataObjects.Net?

Starting from version 4.5 we decided to review our feature planning policy. Not developers or big bosses will decide what to implement first, but you, our dear customers. Especially for this initiative we've opened a forum on excellent UserVoice service to make the voting and suggestion process natural and agile. The forum already contains all suggestions made on support forum or former issue tracker, but you may add your own ideas as well.

For now, there are 2 features in DataObjects.Net 4.5 branch that were planned earlier:
1. Security System, which was requested by many of you.
2. Firebird data provider, which is almost implemented, owing to the considerable contribution of Csaba Beer.

In addition, we will try switching to the recently published PostSharp 2.1 CTP soon in order to check its declared performance boost and overall stability.

We would like to know your opinion on which features we should include into 4.5 branch. There is a possibility to additionally implement one or two big features (depending on their complexity), so make your choice.

DataObjects.Net v.4.4 Beta 2 and v.4.3.7

Development branch

DataObjects.Net 4.4 branch is updated to 4.4 Beta 2 version. This release is mainly dedicated to totally reworked Prefetch module. Changes applied to DataObjects.Net 4.3 branch have been already ported to this version. For further information and usage examples see DataObjects.Net 4.4 Beta 2 Release Notes.

Due to major changes in API, usage of this release in projects, based on the previous version of DataObjects.Net, might require some fixing of assembly references or namespaces. Migration path is described in the Release Notes file.

Download DataObjects.Net 4.4 Beta 2

Stable branch

DataObjects.Net 4.3 branch is updated to 4.3.7 version. This release includes the following changes:

Features:

• Preliminary PostgreSQL 9.0 support
  
• Encrypted connection strings in app.config & web.config
  
• UUID datatype support for PostgreSQL 8.3 and higher
  

Updates:

• Npgsqlprovider is updated to the latest version (2.0.11.0)
  
• More informative descriptions are added to Storage.Model collections
  

Fixes:

• Bug with inherited entities and EntitySet fields
  
• Bug with persistent interfaces and association attributes
  
• Oracle database: Domain.Build throws NotSupportedException
  
• System.InvalidOperationException with message "The ordering of records is not specified for RowNumberProvider"
  
• Take() method generates a top + order by in sql
  
• Exception on ApplyChanges() with MS SQL Server Compact Edition
  
• DateTime validation Issue
  
• Session.ReadRemovedObjects mode is fixed
  
• SessionManager is fixed
  

Misc
DataObjects.Net has been finally reached Microsoft Visual Studio gallery. If you feel that it is worth any positive feedback, please drop a line in Reviews section. Help us making DataObjects.Net popular.

Download DataObjects.Net 4.3.7

How to handle non-persistent collection

Today we've faced with an interesting scenario from Jensen: an Entity has a non-persistent collection which should be serialized to and deserialized from a persistent field on demand. Here is how the problem was initially described on the support website (go to the end of the page):

Object A should contain a list of samples (object T). I could do something like

A a = GetA();
a.Load(); (convert the byte[] to a List)

and something like

a.Store(); (convert the List to a byte[])
m_transactionScope.Commit();

But I don't like calling a Load() and Store() function each time, it would be easier if this could be automated. When A is loaded from the database it should Load(), when saved to the database it should Store().

After several experiments I've found a solution.

1. Add a set of properties to the desired class.

public class MyEntity : Entity {
    ...
    [Field] // Is used as an indicator that collection has been changed
    private bool IsChanged { get; set; }

    [Field] // Backing field for serialized collection value
    private byte[] CollectionValue { get; set; }

    // The collection itself. Note that it is not persistent.
    public ObservableCollection<int> Collection { get; set; }
    ...

2. Add proper methods for serialization & deserialization:

    private static byte[] Serialize(ObservableCollection<int> collection)
    {
      var ms = new MemoryStream();
      var bf = new BinaryFormatter();
      bf.Serialize(ms, collection);
      return ms.ToArray();
    }

    private static ObservableCollection<int> Deserialize(byte[] bytes)
    {
      var bf = new BinaryFormatter();
      var ms = new MemoryStream(bytes);
      return (ObservableCollection<int>) bf.Deserialize(ms);
    }

3. Add Entity initializer & event handlers:

    protected override void OnInitialize()
    {
      base.OnInitialize();
      // Initializing the collection
      if (CollectionValue != null && CollectionValue.Length > 0)
        Collection = Deserialize(CollectionValue);
      else
        Collection = new ObservableCollection<int>();

      // Subscribing to events
      Collection.CollectionChanged += Collection_CollectionChanged;
      Session.Events.Persisting += Session_Persisting;
    }

    void Session_Persisting(object sender, EventArgs e)
    {
      if (!IsChanged)
        return;

      // Serializing the collection right before persist, if it is changed
      CollectionValue = Serialize(Collection);
      IsChanged = false;
    }

    void Collection_CollectionChanged(object sender, NotifyCollectionChangedEventArgs e)
    {
      IsChanged = true;
    }

So, what is going on here?
An instance of MyEntity is subscribed to a CollectionChanged event of MyEntity.Collection and updates MyEntity.IsChanged property when CollectionChanged event is fired. As the property is persistent, the MyEntity instance is added into a queue for persisting. Right before the persist procedure takes place, the instance of MyEntity receives a notification from its Session. During handling of this event we check whether the collection has been changed or not. If so, we update MyEntity.CollectionValue property and changes are successfully persisted to a database.

Note, that I've decided to use ObservableCollection type instead of List because the solution requires a notifications from the collection.

Hope this helps.

Making connection strings secure

Recently, Paul Sinnema from Diartis AG asked on the support website how to make connection strings, listed in application configuration files (app.config or web.config), more secure.

For that time the only thing we could suggest was to use a workaround like that:

1. Encrypt the required connection string with encryption method you prefer and put it somewhere in web.config/app.config file in encrypted form.

2. Load DomainConfiguration through standard API:

var config = DomainConfiguration.Load("mydomain");

3. Set manually config.ConnectionInfo property with decrypted connection string

config.ConnectionInfo = new ConnectionInfo("sqlserver",
  DecryptMyConnectionString(encryptedConnectionString));

4. Build Domain with the config.

Paul answered that although the workaround is useful, he'd prefer that DataObjects.Net could be integrated somehow with standard .NET configuration subsystem that provides easy encrypting/decrypting of connection strings.

Let me explain the approach, Paul was talking about. For example, if plain connection string configuration looks like that;

  <connectionStrings>
     <add name="cn1" connectionString="Data Source=localhost\SQL2008;Initial Catalog=DO40-Tests;Integrated Security=True;MultipleActiveResultSets=True" />
  </connectionStrings>

then after executing "aspnet_regiis.exe -pef "connectionStrings" %PATH_TO_PROJECT%" the encrypted one will look like the following:

  <connectionStrings configProtectionProvider="RsaProtectedConfigurationProvider">
    <EncryptedData Type="http://www.w3.org/2001/04/xmlenc#Element"
      xmlns="http://www.w3.org/2001/04/xmlenc#">
      <EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#tripledes-cbc" />
      <KeyInfo xmlns="http://www.w3.org/2000/09/xmldsig#">
        <EncryptedKey xmlns="http://www.w3.org/2001/04/xmlenc#">
          <EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#rsa-1_5" />
          <KeyInfo xmlns="http://www.w3.org/2000/09/xmldsig#">
            <KeyName>Rsa Key</KeyName>
          </KeyInfo>
          <CipherData>
            <CipherValue>TJETPde0GVrSAKk0VpUs3EsV8XFRQrVU1lGlrX8aEhvfa4MaMpN3O6KLo3I9e5oujXBImKtmXHWxnF6KRcSWteuQXD4I37Dq6yE1cZ4qDHqNMkcQK+lgi6jbRLcPzYQdFuOLagsAVpif8OvY0dJkiKd+Srqjwz7ZO225VCipaQs=</CipherValue>
          </CipherData>
        </EncryptedKey>
      </KeyInfo>
      <CipherData>
        <CipherValue>6bQyCUdH9rqdNAD4IwmSXtAQIEuRfSjpQ/Py5ovz2ZpwJdEXa4W3OKWMm52g1ARXPqMYw1thfQae//+1JWTAPQwrE3JTZEdcgu47KE2x8uqdWeJfoXSecp6berSPc4qAa4F2B5x8weUGPiAsBldVfBXwUBPniKWbnnXViJWmcSn16kaAtIkq5GtNOp7YhPt9WKZG0QkFRpEaRAWqLgxK6nXUwK8BgbGEBI4B27QSFj2ME1/JI+o//RrchntEuW6VolEA7PAejMujsIn1xb0hqg==</CipherValue>
      </CipherData>
    </EncryptedData>
  </connectionStrings>

As a result, the connection strings get encrypted and therefore, much more secure. Working with them is as simple as usual: they can be accesses through standard ConfigurationManager without any special tricks:

ConfigurationManager.ConnectionStrings["cn1"].ConnectionString

And this simplicity makes the whole approach extremely useful.

So, what could be done to get DataObjects.Net compatible with all this shiny stuff, keeping in mind that connection strings section is outside DataObjects.Net configuration?

After thinking for a while on the problem, we've got the following idea: what if we put in domain configuration not a connection string itself, but a reference to the connection string, that is actually defined in connectionStrings section? For instance:

<Xtensive.Orm>
    <domains>
      <domain provider="sqlserver" connectionString="#cn1" ... />
    </domain>
  </domains>
<Xtensive.Orm>

Where:

• cn1 is a name of a connection string defined in connectionStrings section;
• # sign is used to inform that actually this is a reference to a connection string and DataObjects.Net must resolve it through ConfigurationManager.

Having this feature implemented, DataObjects.Net will provide support for encrypted connection strings out-of-the-box.

And now, we'd like to ask you some important questions:

• What would you say about the feature?
• Will it be useful for you and should it be implemented?
• Is the "#name" notation for setting a reference to a connection string is more or less clear or wouldn't it better to change it to something else?

Dear users of DataObjects.Net, we need your feedback. Please, post a comment or two.

Thanks in advance.

Christmas Sale

We are glad to announce that Xtensive.com Christmas Sale has started today, December 15 and will last till the end of the year. During this time you get 20% off any edition of DataObjects.Net and Help Server. To apply this 20% Christmas special to your order, just enter the following coupon code at the shopping cart or product checkout page: DEC2010PROMO

Should you have any questions, please email to sales@x-tensive.com

Merry Christmas and Happy New Year!

Sincerely,
DataObjects.Net Team

DataObjects.Net v.4.4 Beta, v.4.3.6 and the Christmas Sale

Development branch

Today we are happy to announce the next significant step in DataObjects.Net 4 development — DataObjects.Net 4.4 Beta 1. This release includes the following exciting features and updates:

• ORM platform unification for usage in both server-side and client-side applications
• Optional Session auto-activation
• Optional Transactions
• 'Unit of work' pattern support in UI applications
• Greatly reduced number of assemblies (only 3 assemblies instead of 17)
• F# language support
• Visual Basic language support
• Xtensive Practices initiative
• and much more...

For further information see DataObjects.Net 4.4 Beta 1 Release Notes.

Due to major changes in API, usage of this release in projects, based on the previous version of DataObjects.Net, might require some fixing of assembly references or namespaces. Migration path is described in the Release Notes file.

Download DataObjects.Net 4.4 Beta 1

Stable branch

DataObjects.Net 4.3 branch is updated to 4.3.6 version. This release includes the following changes:
Features

• Visual Basic support

Updates

• EntitySet now implements IValidationAware
• SessionOptions.ReadRemovedObjects is added

Bug fixes

• struct serialization bug
• OrderBy(...).ThenBy(...)
• ReferenceFinder.IsReferenced throws an exception
• 'is' operator doesn't work in LINQ
• Error on schema upgrade with inheritance change (type removal)
• CopyFieldHint \ MoveFieldHint does not work when reference field is moved in hierarchy
• Mutithreading exception on Prefetch
• Nanoseconds handling in MS SQL Server 2005
• DateTime handling in version fields in Ms SQL Server 2005
• FreeText in PostgreSQL
• Grouping by nullable date in PostgreSQL
• StoreProvider is fixed  in PostgreSQL
• Referential integrity violation message improvement
• Wrong FK creation behavior & association inheritance
• [Field(LazyLoad = true)] is ignored by some code executed during Entity.Remove()
• Translation of DateTime.Date for SqlServer family

Download DataObjects.Net 4.3.6

And one more thing

Xtensive LLC announces the Christmas Sale, which is starting in the middle of December.
Look for Christmas coupons on our websites, take your chance and get 20% off regular product price.

Stay tuned.