SQL Server Compact Private Deployment tweaks

As a follow up to my previous post about Private Deployment (the concept that you can simply include the SQL Server Compact DLL files with your application as content, so to speak), I will show you a couple of tweaks that the .NET Framework enables.

Forcing an existing application to use the private DLL files

If you have an existing application, that is compiled against the centrally deployed DLL files, for example assembly version 3.5.1.0 or 4.0.0.0, you can force the application to use the private assembly version files instead (3.5.1.50 and 4.0.0.1), via an application configuration file. Lets take ExportSqlCe40.exe as an example. This application is complied against assembly version 4.0.0.0, so it will not work unless SQL Server Compact 4.0 runtime is centrally installed.

To force this application to use Private Deployment only, create a .config file named ExportSqlCe40.exe.config, with the following contents:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <runtime>
    <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
      <dependentAssembly>
        <assemblyIdentity name="System.Data.SqlServerCe" publicKeyToken="89845dcd8080cc91" />
        <bindingRedirect oldVersion="4.0.0.0" newVersion="4.0.0.1" />
      </dependentAssembly>
    </assemblyBinding>
  </runtime>
</configuration>

If you run the application now, you will get this error:

Now copy all files from the C:\Program Files\Microsoft SQL Server Compact Edition\v4.0\Private folder to the folder where the Exportsqlce40.exe file resides:

Now the application runs, and uses only the private DLL files.

Isolating the SQL Server Compact runtime files in a separate folder

Continuing the sample above, to be neater, it would be nice to have the SQL Server Compact DLL files in a subfolder below the .exe file location. This can be done by moving the files to a separate folder, for example named SqlCe4. Now I have moved all the SQL Server Compact files and folders to that folder:

Now modify the .config file as follows:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <runtime>
    <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
      <probing privatePath="SqlCe4"/>
      <dependentAssembly>
        <assemblyIdentity name="System.Data.SqlServerCe" publicKeyToken="89845dcd8080cc91" />
        <bindingRedirect oldVersion="4.0.0.0" newVersion="4.0.0.1" />
      </dependentAssembly>
    </assemblyBinding>
  </runtime>
</configuration>

Notice the Probing setting above, that has been added. Also notice that this comes before the bindingRedirect.

Hop you find this useful.

SQL Server Compact Toolbox 2.2–Visual Guide of new features

After more that 32.000 downloads, version 2.2 of my SQL Server Compact Toolbox extension for Visual Studio 2010 is now available for download. This blog post is a visual guide to the new features included in this release, many suggested by users of the tool via the CodePlex issue tracker

Generate an Windows Phone DataContext (.cs) in the current project (beta feature)

This feature allows you to create a DataContext for use with Windows Phone “Mango” projects, based on an existing SQL Server Compact 3.5 database.

This will be a huge timesaver, and allows you to reuse the effort you may already have put in creating your database schema, including foreign keys and the required indexes. It will also save you much repetitive code, as the generated classes reflect your tables 1:1.

This technique even works with Northwind.sdf, despite object names with spaces. None of this is (of course) supported in any way by Microsoft.

Under the covers, I am using SQLMetal to generate a desktop DataContext class, and then this class is enhanced to work well in a Windows Phone project.

Advantages of this approach:
- Use desktop database tools for data population and schema design
- Saves time doing 1:1 mapping between database tables and DataContext classes
- DataContext classe and entity classes are partial and can be expanded
- Invalid DataContext constructors removed
- The generated DataContext contains Index definitions (which SqlMetal does not support, as this is a Windpows Phone extension)
- The generated DataContext contains the CreateIfNotExists method, that optionally extracts an included database (prepopulated with data) to Isolated Storage

If you also would like to include your desktop database with your XAP, the generated DataContext contains the code from this blog post and supports the procedure described in the blog post.

Please provide any feedback for this beta feature to the CodePlex issue tracker.

Add Description to tables and columns

On Database, Table and Column level, there is a new menu item: Edit Description, that allows you to enter a object description. The object description are stored in a table named __ExtendedProperties in the current database.

The description is then shown as a tooltip for the object:

In addition, you can reuse the data for documentation etc. (I am planning a documentation feature in a future release)

Options dialog for saving options from session to session

A new options dialog is available from the main toolbar:

SQL Editor improvements

The SQL Editor (again) allows you to display the results in a Grid, rather than text. This can be selected via the Options above. (Using grid will be slower and requires more memory). Also, a Save Script button has been added.

Other minor improvements

Latest scripting libraries included, with improvements to Db Diff and primary keys with multiple columns

Thanks to all Toolbox users for your continued encouragement and feedback!

Populating a Windows Phone “Mango” SQL Server Compact database on desktop

For up to the minute news, outcries, complaints and cheers on SQL Server Compact, follow me on Twitter: @ErikEJ

If you want to prepopulate a Mango SQL Server Compact database with some data, you can use the following procedure to do this. (Notice, that the Mango tools are currently in beta)

First, define your data context, and run code to create the database on the device/emulator, using the CreateDatabase method of the DataContext. See a sample here. This will create the database structure on your device/emulator, but without any initial data.

Then use the Windows Phone 7 Isolated Storage Explorer to copy the database from the device to your desktop, as described here.

You can now use any tool, see the list of third party tools on this blog, to populate your tables with data as required. The file format is version 3.5. (not 4.0)

Finally, include the pre-populated database in your WP application as an Embedded Resource.

UPDATE: I was made aware, that for read only data, you can just include the database file as Content (not Embedded Resource), and it will be available from the special appdata: URI, with a connection string like the following:

“Data Source=appdata:/Chinook.sdf;Mode=Read Only”

So no need to run code to extract from an Embedded resource as below in that case.

You can then use code like the following to write out the database file to Isolated Storage on first run:

public class Chinook : System.Data.Linq.DataContext
{
    public static string ConnectionString = "Data Source=isostore:/Chinook.sdf";

    public static string FileName = "Chinook.sdf";

    public Chinook(string connectionString) : base(connectionString) { }

    public void CreateIfNotExists()
    {
        using (var db = new Chinook(Chinook.ConnectionString))
        {
            if (!db.DatabaseExists())
            {
                string[] names = this.GetType().Assembly.GetManifestResourceNames();
                string name = names.Where(n => n.EndsWith(FileName)).FirstOrDefault();
                if (name != null)
                {
                    using (Stream resourceStream = Assembly.GetExecutingAssembly().GetManifestResourceStream(name))
                    {
                        if (resourceStream != null)
                        {
                            using (IsolatedStorageFile myIsolatedStorage = IsolatedStorageFile.GetUserStoreForApplication())
                            {
                                using (IsolatedStorageFileStream fileStream = new IsolatedStorageFileStream(FileName, FileMode.Create, myIsolatedStorage))
                                {
                                    using (BinaryWriter writer = new BinaryWriter(fileStream))
                                    {
                                        long length = resourceStream.Length;
                                        byte[] buffer = new byte[32];
                                        int readCount = 0;
                                        using (BinaryReader reader = new BinaryReader(resourceStream))
                                        {
                                            // read file in chunks in order to reduce memory consumption and increase performance                    
                                            while (readCount < length)
                                            {
                                                int actual = reader.Read(buffer, 0, buffer.Length);
                                                readCount += actual;
                                                writer.Write(buffer, 0, actual);
                                            }
                                        }
                                    }

                                }
                            }
                        }

                        else
                        {
                            db.CreateDatabase();
                        }
                    }
                }
                else
                {
                    db.CreateDatabase();
                }
            }
        }
    }
}

SQL Server Compact ASP.NET Membership, Role and Profile Provider version 2.1 now available

My ASP.NET membership provider is now available in version 2.1, that contains many improvements and some new features based on excellent community feedback – keep it coming!

The ASP.NET membership provider project was prompted last July by the comments to Scott Gu’s blog post about the upcoming version 4.0 of SQL Server Compact, and it’s support for ASP.NET.

Basically the Gu said: “We are looking to potentially ship a set of providers that work with it (and do not use stored procedures). The first beta won't have this - but it is something we'll hopefully enable in the future.”

So it was time to start coding, since the absence of a Membership provider would make SQL Server Compact less of an attractive option for ASP.NET web sites.

Since then, the database schema used has been refactored to be in line with the ASP.NET 4.0 SQL Server based schema, which resulted in the first NuGet Package being released in January 2011.

Now version 2.1 is available, also via NuGet:

Or from the CodePlex site.

The new features in version 2.1 are:
Profile provider included (contrib davidsk)
Two new methods: UpdateUserName and MigrateMembershipDatabaseToAspNet40 (contrib nekno)

Bug fixes (by various contributors, thank you all):
UpdateUser() doesn't set LoweredEmail
GetUser w/ providerUserKey returns invalid information
Static salt leads to deterministic output, dynamic salt is better
Configuration error when using a provider

Scripting image column data for INSERT statements

I earlier blogged about scripting datetime values for INSERT statements, or for example by use in the SubmitSQL method of the SQL Server Compact RemoteDataAccess API. In this post, I will show how to serialize byte arrays as a hexadecimal string. Notice that for the SubmitSQL method, there is a limit to the size of the SQL statement (not sure what is is, but 64 K is a good guess, I think). So if you have large images, that you want to send to your server, you are out of luck with this method.

Below is the code I currently have implemented in my ExportSqlCE INSERT statement generator.

if (dt.Columns[iColumn].DataType == typeof(Byte[]))
{
    Byte[] buffer = (Byte[])dt.Rows[iRow][iColumn];
    _sbScript.Append("0x");
    for (int i = 0; i < buffer.Length; i++)
    {
        _sbScript.Append(buffer[i].ToString("X2", System.Globalization.CultureInfo.InvariantCulture));
    }
}

Notice the special “X2” string format, this is what performs the magic of creating the hex string. Prepending the string with 0x is required by SQL Server. (_sbScript is a StringBuilder)

There is a long discussion on Stackoverflow on how to do this faster, the code below seems to be the fastest:

private static string ByteArrayToHex(byte[] barray)

{


char[] c = new char[barray.Length * 2];


byte b;


for (int i = 0; i < barray.Length; ++i)


{


b = ((byte)(barray[i] >> 4));


c[i * 2] = (char)(b > 9 ? b + 0x37 : b + 0x30);


b = ((byte)(barray[i] & 0xF));


c[i * 2 + 1] = (char)(b > 9 ? b + 0x37 : b + 0x30);


}


return new string(c);


}




Maybe I should update my ExportSqlCe code? - Happy scripting!

SQL Server Compact Toolbox standalone - including all managed DLLs in a single .exe

The latest release of the standalone version of my SQL Server Compact Toolbox, mainly for users that do not have Visual Studio 2010 Pro or higher, is available as a single .exe. It was actually a Tweet from @scottgal, that pointed me towards this excerpt from Jeffery Richters’ CLR via C#, Third Edition.

In order to implement in the WPF application, that is the standalone Toolbox, I added the following code to App.xaml.cs (and a Startup handler to App.xaml):

private void Application_Startup(object sender, StartupEventArgs e)
{
    AppDomain.CurrentDomain.AssemblyResolve += (ssender, args) =>
    {
        //string[] names = this.GetType().Assembly.GetManifestResourceNames();

        String resourceName = "ErikEJ.SqlCeToolbox." +
           new AssemblyName(args.Name).Name + ".dll";

        using (var stream = Assembly.GetExecutingAssembly().GetManifestResourceStream(resourceName))
        {
            Byte[] assemblyData = new Byte[stream.Length];
            stream.Read(assemblyData, 0, assemblyData.Length);
            return Assembly.Load(assemblyData);
        }

    }; 
}

I also added all the Managed libraries that the Toolbox uses as Embedded Resources.

I use the following libraries, all from CodePlex:

WPF Property Grid 
http://wpg.codeplex.com/ (for the SqlCeConnectionStringBuilder)
Sample usage

KBCsv 
http://kbcsv.codeplex.com/ (for .csv file import)
Sample usage

SQL Server Compact Scripting Library (for database scripting) 
http://exportsqlce.codeplex.com/
Sample usage Sample usage

FabTab WPF Tab Control (the SQL Editor tabs)
http://fabtab.codeplex.com/
Sample usage

And the Microsoft Data Connection Dialog (to prompt fro a SQL Server Connection) from http://archive.msdn.microsoft.com/Connection
Sample usage

Hope you find this tip useful.

New release of Scripting Library and command line utilities–with Schema Diff and DGML from command line

The latest release of my ExportSqlCe SQL Server Compact scripting library and related command line utilities is now available on CodePlex.

This latest version of the command line utilities adds the capability to generate Schema Diff and DGML database graph files.

The schema diff option allows you to compare a SQL Server Compact database file with another SQL Server Compact database file or even a SQL Server database, and creates a script with the required ALTER TABLE etc. statements to synchronize the 2 database schemas.

The DGML option allows you to create a graphical view of the database tables and fields, the resulting .dgml file requires Visual Studio 2010 Premium or higher to be viewed. I blogged about DGML files earlier:

http://erikej.blogspot.com/2010/04/diagram-database-table-relationships.html

http://erikej.blogspot.com/2010/08/sql-server-compact-35-toolbox-updated.html

The latest documentation for the command line utilities is available here.

And both these file types can of course be generated from your own application, using the scripting library. I have some code samples available here.