SQL Server Compact ADO.NET data access performance – part 2: INSERTs

In this second part of this series, I will show by examples how to use the two available ADO.NET APIs in the SQL Server Compact ADO.NET provider for INSERTing data. I will also show some informal performance measurements, but keep in mind that your scenario may give different results.

In the sample solution, I will create a data access library for maintaining the following table, which you could be imagine could be used in a caching library:

CREATE TABLE CacheElement (
    [Key] uniqueidentifier NOT NULL,
    Tag NVARCHAR(4000) NULL,
    Value image NOT NULL,
    CreatedAt DATETIME NOT NULL,
    ExpirationAt DATETIME NOT NULL);

ALTER TABLE [CacheElement]
    ADD CONSTRAINT [PK_CacheElement] PRIMARY KEY ([Key]);

This table will represent the following class:

public class CacheElement
{
    public Guid Key { get; set; }
    public string Tag { get; set; }
    public Byte[] Value { get; set; }
    public DateTime CreatedAt { get; set; }
    public DateTime ExpirationAt { get; set; }
}

I will implement the following interface methods in the two possible ways, and then compare implementation and timings, by calling the library from a Unit test project:

    public interface IDataAccess
    {
        void InsertElement(CacheElement element);
        void InsertElements(List<CacheElement> elements);
    }

(I will add methods and implement further in the upcoming blog posts)

First let us look at the implementation of the T-SQL based INSERT, using best practices with explicit parameters (in the ClassicDal class) :

public void InsertElement(CacheElement element)
{
    using (var command = _connection.CreateCommand())
    {
        command.Parameters.Add("Key", SqlDbType.UniqueIdentifier).Value = element.Key;
        command.Parameters.Add("Value", SqlDbType.Image).Value = element.Value;
        command.Parameters.Add("CreatedAt", SqlDbType.DateTime).Value = element.CreatedAt;
        command.Parameters.Add("ExpirationAt", SqlDbType.DateTime).Value = element.ExpirationAt;
        if (String.IsNullOrWhiteSpace(element.Tag))
        {
            command.Parameters.Add("Tag", SqlDbType.NVarChar, 4000).Value = DBNull.Value;
        }
        else
        {
            command.Parameters.Add("Tag", SqlDbType.NVarChar, 4000).Value = element.Tag;
        }

        command.CommandText = @"INSERT INTO CacheElement
            ([Key],Tag,Value,CreatedAt,ExpirationAt)
            VALUES   
            (@Key, @Tag, @Value, @CreatedAt, @ExpirationAt)";
        command.ExecuteNonQuery();
    }
}

For all tests, I am using best practices for SQL Compact connection handling, and passing an already open connection. This avoids measuring the overhead of loading the SQL Compact engine DLL files, and opening the database file. Notice that for NULLable values, special handling has to be made.

Now lets us look at the implementation that uses the “raw” APIs for INSERTs (in the RawDAL class) :

public void InsertElement(CacheElement element)
{
    using (var command = _connection.CreateCommand())
    {
        command.CommandType = CommandType.TableDirect;
        command.CommandText = "CacheElement";
        using (var resultSet = command.ExecuteResultSet(ResultSetOptions.Updatable))
        {
            SqlCeUpdatableRecord record = resultSet.CreateRecord();
            record.SetGuid(0, element.Key);
            if (String.IsNullOrWhiteSpace(element.Tag))
            {
                record.SetValue(1, DBNull.Value);
            }
            else
            {
                record.SetString(1, element.Tag);
            }
            record.SetBytes(2, 0, element.Value, 0, element.Value.Length);
            record.SetDateTime(3, element.CreatedAt);
            record.SetDateTime(4, element.ExpirationAt);
            resultSet.Insert(record);
        }
    }
}

Notice the following special pattern: Setting the CommandType to TableDirect, the CommandText is the table name, we use the CreateRecord() method to get a SqlCeUpdateableRecord with “slots” that match our table columns. You have to know the exact “ordinal position” of your columns, you can get that by scripting a CREATE TABLE statement with my Toolbox, or inspecting the INFORMATION_SCHEMA.COLUMNS table. There are typed SetXxx methods that must match the datatype of your columns. Finally, call the Insert method to add the “record”.

In the sample code, I have also implemented methods to insert many rows in a single method invocation, thus saving the overhead of recreating a number of objects for each INSERT. This is similar to the way I insert data with my SqlCeBulkCopy library.

First the timings for a single INSERT: Raw: 31 ms, Classic: 29 ms – hardly any difference. Then 1000 single INSERTs, minimal object reuse: Raw: 2548 ms, Classic: 1936 – in this case not any advantage of the raw api. Then finally 1000 INSERTs, maximum object reuse: Raw: 73 ms, Classic: 354 ms. A significant difference, if that is a pattern you have in your application, ie inserting many rows in a single call.

You can download the code sample from here, and stay tune for the next installment: SELECT (using SetRange and Seek)

SQL Server Compact ADO.NET data access performance – part 1: Overview

In this five part series on SQL Server Compact performance, I will demonstrate and measure the performance of the two available data access APIs provider by the SQL Server Compact ADO.NET provider (System.Data.SqlServerCe.dll). The information in this blog post applies to both version 3.5 and 4.0. I will not cover usage of the unmanaged OLEDB interfaces (using C and C++)

As mentioned, the ADO.NET provider provides two APIs for data access (CRUD) operations:

One: The T-SQL standards based APIs - SqlCeCommand: ExecuteReader, ExecuteNonQuery, ExecuteScalar, that allows you to use text based SQL statements, including parameterized SQL. Most ORM implementations that support SQL Server Compact, like LINQ to SQL and Entity Framework, use this API, with the exception of the high perfromant OpenNETCF.ORM Framework, which is able to take advantage of

Two: The “table direct” APIs that allow you to bypass the T-SQL parsing and interpretation process, and that alone provides an obvious saving in processing. These include:

SqlCeResultSet (that inherits from SqlCeDataReader) - in particular when used with a SqlCeCommandType of TableDirect

SqlCeUpdateableRecord

SqlCeCommand.ExecuteResultSet

SqlCeDataReader.Seek

SqlCeCommand.SetRange

As you can see, none of these is available with the SQL Server ADO.NET objects, and they are particular to SQL Server Compact. This blog post series will demonstrate how to use these as alternatives to the “standard” methods.

The diagram from Laxmi’s blog post here explains the advantage graphically:

The “table direct” APIs bypass the Query Processor layer, and instead uses an ISAM like access pattern.

In the next blog post in this series, I will compare INSERT with SqlCeResultSet.Insert( SqlCeUpdateableRecord ) and demonstrate how to use the “Table Direct” APIs for INSERTs.