Windows Phone Local Database tip: Viewing the SQL generated by LINQ to SQL

It may sometimes by useful to be able to see the SQL statements generated by an ORM, in order to understand better what is going on, and why things perform the way they do. With LINQ to SQL on desktop, you can assign any TextWriter object to the DataContext Log property, and this will allow you to log to a file or the console.

On Windows Phone, this gets a little bit complicated, as files must be stored in Isolated Storage, making them hard to get at, and there is no Console. This MSDN blog includes a TextWriter implementation, that writes to the console of a attached debugger, for example the Visual Studio Debug output Window. This class (which I have renamed to DebugWriter) will be part of the DataContext generated by the next SQL Server Compact Toolbox release. So now you can add the following while debugging your LINQ to SQL statements:

db.Log = new DebugWriter();

We can now try some various LINQ queries against our database to find out how they have been implemented in SQL (I am using the excellent Chinook sample database, available for download here):

db.Album.Where(al => al.AlbumId == 10).SingleOrDefault();

And the output in the Debug output window:

db.Album.FirstOrDefault(); 

Becomes:

SELECT TOP (1) [t0].[AlbumId], [t0].[Title], [t0].[ArtistId]
FROM [Album] AS [t0]

db.Album.Where(al => al.Title.StartsWith("For those%")).FirstOrDefault();

Becomes:

SELECT TOP (1) [t0].[AlbumId], [t0].[Title], [t0].[ArtistId]
FROM [Album] AS [t0]
WHERE [t0].[Title] LIKE @p0

db.Album.Skip(50).Take(25).ToList();

Becomes:

SELECT [t0].[AlbumId], [t0].[Title], [t0].[ArtistId]
FROM [Album] AS [t0]
ORDER BY [t0].[AlbumId] OFFSET @p0 ROWS FETCH NEXT @p1 ROWS ONLY

Notice that the last statement uses OFFSET and FETCH, a syntax only introduced in SQL Server Compact 4.0, but apparently retrofitted into the version 3.5 engine running on Windows Phone.

Happy SQL debugging!

SQL Compact 3rd party tools

This is an attempt at a comprehensive list of SQL Server Compact Microsoft and 3rd party tools, both commercial and non-commercial. Please let me know if I have missed anything, and I will add it.

Microsoft supplied tools

Visual Studio Community 2017 free – when used with the SQL Server Compact / SQLite Toolbox: SQL CE 3.5, SQL CE 4.0

Visual Studio 2008 Professional (SQL CE 3.5)

Visual Studio 2005 Standard (SQL CE 3.1)

SQL Server 2008 R2 Management Studio Express (free, SQL CE 3.5 SP2)
Link: http://www.microsoft.com/en-us/download/details.aspx?id=30438
SQL Server 2005 Management Studio Express SP2 (free, SQL CE 3.1)
Link: http://www.microsoft.com/en-us/download/details.aspx?id=15366

New or updated entries are noted with (new) or (updated) ”Last updated” indicates when the tool was last updated by it’s author.

Non-commercial (free) tools

My GitHub Tools

Name: SQL Server Compact Toolbox (updated)
Link: https://github.com/ErikEJ/SqlCeToolbox
SQL CE Versions: 3.5, 4.0
Last updated: Current
Platforms: Win32, Win64 
Added to list: 29th June 2010
”Add-in for Visual Studio 2071/2015/2013/2012 (for 3.5 and 4.0). Standalone app (for 4.0 and 3.5)
Adds several features to help your SQL Server Compact development efforts:
- Scripting of tables, including data.
- Script of entire schema, optionally with data, both of SQL Server Compact and SQL Server 2005 or later databases, including SQL Azure.
- Import to SQL Compact 3.5 from a server database or CSV file.
- Basic, free form query execution.
- Generation of DGML files for visualizing table relationships
- Database Schema Diff Script
- Detect database file version
- Fix broken connection definitions”

Name: SqlCeScripting.dll (updated)
Link: https://github.com/ErikEJ/SqlCeToolbox  
SQL CE Versions: 3.5, 4.0
Last updated: Current
Platforms: Win32, Win64 
Added to list: 29th June 2010
”Scripting library for inclusion in your own application, enumerates SQL Compact objects and allows scripting of these.”

Name: SqlCeCmd (updated)
Link: https://github.com/ErikEJ/SqlCeCmd
SQL CE Versions: 3.5, 4.0
Last updated: 2015
Platforms: Win32, Win64
”Administer SQL Compact databases directly from the command line, without having Visual Studio or SQL Server Management Studio installed. Only requires NetFx 2.0 and SQL Compact installed. Command line options similar to sqlcmd. Runs scripts generated by ExportSqlCe.”

Name: ExportSqlCe (updated)
Link: https://github.com/ErikEJ/SqlCeToolbox/
SQL CE Versions: 3.0, 3.5, 4.0
Last updated: Current
Platforms: Win32, Win64 
”This command line utility helps you better manage your SQL Compact development efforts. If allows you to script schema and data from any SQL Compact database file to a .sql file, which can be used in any context. The utility is developed in C# with Visual Studio 2008. You can use the resulting script for documentation, programmatic scripting of SQL Compact database files, or migration of schema and data to SQL Server (Express) 2005 or later.

Name: Export2SqlCe (updated)
Link: https://github.com/ErikEJ/SqlCeToolbox
SQL Server Versions: 2005, 2008, 2012, 2014 
Last updated: Current
Platforms: Win32
Added to list: 4th March 2010
”A command line utility to script an entire SQL Server 2005 or later database with or without data to a SQL Compact T-SQL script.”

Name: SqlCeBulkCopy (updated) 
Link: https://github.com/ErikEJ/SqlCeBulkCopy
SQL CE Versions: 3.5, 4.0
Last updated: Current 
Platforms: .NET library, .NET CF library 
Added to list: 4th March 2010 
”.NET Library for loading data fast (doing bulk inserts) into a SQL Server Compact database file. Attempts to mimic the SQLClient SqlBulkCopy API. Includes support for BULK INSERTing any IEnumerable”

Query and database management tools

Name: CompactView
Link: http://sourceforge.net/p/compactview/home/Home/
SQL CE Versions: 3.1, 3.5, 4.0
Last updated: 2015
Platforms: Win32, Win64
Added to list: 14th November 2011
CompactView is a viewer for Microsoft® SQL Server® Compact Edition (SQLCE) database files (*.sdf). CompactView can open database files of versions 3.1, 3.5 and 4.0

Name: SQL Compact Query Analyzer
Link: http://sqlcequery.codeplex.com/
SQL CE Versions: 3.1, 3.5, 4.0
Last updated: 2013
Platforms: Win32, Win64
Added to list: 3rd July 2011 
Features:
- Execute SQL Queries against a SQL Server Compact Edition database
- Table Data Editor to easily edit the contents of the database
- Supports SQLCE 3.0, 3.1, 3.5 and 4.0
- Execute multiple SQL queries (delimited by a semi colon ;)
- Display query result as XML
- Shrink and Compact Databases
- Generate Schema and Data Scripts
- Display database and schema information

Name: LINQPad
Link: http://www.linqpad.net/
SQL CE Versions: 3.5, 4.0
Last updated: Current
Platforms: Win32, Win64
Added to list: 14th December 2010
Linqpad allows you to query various data sources via LINQ, including SQL Server Compact 3.5 and 4.0. See this blog post.

Name: Database .NET
Link: http://fishcodelib.com/Database.htm
SQL CE Versions: 3.1, 3.5, 4.0
Last updated: Current
Platforms: Win32, Win64
Added to list: 20th September 2010
“Database .NET is a simple and intuitive database management tool for multiple databases. Making Local and Remote Database Administration Easier, With it you can Create databases, Design tables, Edit data, Export data, Generate scripts and run SQL queries.”

Name: SQL CE Browser
Link: http://ericwillis.com/notes/2009/12/sql-ce-browser-v-110/
SQL CE Versions: 3.5
Last updated: 2010
Platforms: Win32, Win64 (?)
Added to list: 4th March 2010 
”Open regular and encrypted/passworded databases for viewing or editing. Write and work with any number of queries at once via tabs. SQL syntax highlighting and SQL syntax verification. Explore columns, keys, and indexes“

Name: Mini SQL Query
Link: http://minisqlquery.codeplex.com/ 
SQL CE Versions: 3.5
Last updated: 2014
Platforms: Win32, Win64 (?)
Added to list: 4th March 2010 
”Mini SQL Query is a minimalist SQL query tool for multiple databases (MSSQL, MSSQL CE, SQLite, OLEDB, MS Access/Excel files etc). The tool uses a generic schema engine taking advantage of the ADO.NET providers. Mini SQL Query is also easily extended with plugins.“

Name: SQL Server CE Query Tool
Link: http://cequery.codeplex.com
SQL CE Versions: 3.5, 4.0
Last updated: 2011 
Platforms: Win32, Win64 (?)
”CEQuery is written in C# with the help of Visual Studio 2008 platform. Some of the features of this tool are described below:
User is able to create CE database. If the schema is provided, the tool is able to create the tables and columns. User is able to overview an existing CE database with its table and column structure in tree format. User can design a query by dragging and dropping tables and columns from the database tree. User can manipulate [Insert/Update/Delete] records in a selected table. User is able to generate scripts from a single table or of whole databases. [Both for SQL CE or SQL Server]. User is able to open multiple SQL Server CE files simultaneously. User is able to convert a SQL Server database to a SDF file with schema and data. User is able to change password of a SQL CE DB. The tool is now capable of handling the database with no password.“

Name: SeasonStar Database Management(SSDM)
Link: http://ssdm.codeplex.com/
SQL CE Versions: 3.5?, 4.0 
Last updated: 2011
Platforms: Win32
”With support for all major database formats, top-class export functions and advanced feaures like Linq Data Context availabe, SSDM is probably your best choice for DB management. It's even free software!”

Name: SQL CE Database Editor
Link: http://sqlcedatabaseeditor.codeplex.com/
SQL CE Versions: 3.5
Last updated: 2009 
Platforms: Win32
“Small lightweight application to browse and edit Microsoft SQL CE server databases. Does not require SQL CE server to be installed and can run custom queries.”

Name: DataBoy
Link:http://databoy.codeplex.com/
SQL CE Versions: 3.0, 3.5
Last updated: 2008
Platforms: Win32
“Small footprint, portable and simple database query tool. Support SQL CE 3.1, SQL CE 3.5 and SQL Server.”

Other tools and libraries

Name: Lyare.SqlServerCe.MultiQuery (new)
Link: https://github.com/lokiworld/Lyare.SqlServerCe.MultiQuery
SQL CE Versions: 4.0
Last updated: 2014
Platforms: Win32, Win64
Added to list: 2nd March, 2014 
“An extension to the Microsoft SQL Server Compact Edition client library, which simulates the support of the multiple statements in a single command.”


Name: SQLCE SCRIPTER
Link: http://sqlmgmt.com/sql/SQLCE-Scripter
SQL CE Versions: 3.5, 4.0
Last updated: 2011
Platforms: Win32, Win64
Added to list: 14th November, 2011
“Create SQL schema scripts for tables, indexes and referential relations. Create data scripts so that CE data can be moved to SQL Server or other data store. User can select specific tables only. User can select Script Schema, Script Data or both.”

Name: SQLCE Code Generator
Link: http://sqlcecodegen.codeplex.com/
SQL CE Versions: 3.5, 4.0
Last updated: 2012 
Platforms: Win32, Win64
Added to list: 27th March, 2011
“Contains a stand alone GUI application and a Visual Studio Custom Tool for automatically generating a .NET data access layer code with Unit tests for objects in a SQL Server Compact Edition database. Also generates a Windows Phone DataContext”

Name: OpenNETCF.ORM Framework
Link: http://orm.codeplex.com/
SQL CE Versions: 3.5
Last updated: Current 
Platforms: WinCE, Win32, Win64
Added to list: 27th March, 2011
“The OpenNETCF.ORM library is a lightweight object-relational mapping framework designed for use with the .NET Compact Framework.The framework is extensible to allow any type of backing store. An initial implementation using SQL Server Compact is provided.”

Name: SQL Metal Open Source User Interface
Link: http://sqlmetalosui.codeplex.com/
SQL CE Versions: 3.5
Last updated: 2008 
Platforms: Win32
“This project provides an user interface for SqlMetal.exe tool.
SQLMetalOSUI allows users to generate code and mapping files for their SQL Server Compact Edition databases. SQL Server Compact Edition 3.5 is fully compatible with LINQ to SQL but the designer does not. This project will be focused to reach this initial purpose.”

Commercial tools

Name: MS Compact Maestro (new)
Link: http://www.sqlmaestro.com/products/mssql/compact_maestro/ 
SQL CE Versions: 3.5, 4.0
Last updated: 2013
Platforms: Win32, Win64
”MS Compact Maestro is a top-class SQL Server Compact management and development tool.”

Name: SDF Viewer 
Link: http://www.flyhoward.com/SDF_Viewer.aspx
SQL CE Versions: 3.0, 3.5, 4.0
Last updated: Current
Platforms: Win32
“The simple way to create, view, edit or export the data in your Windows Mobile or Pocket PC device's SQLCE database, from the convenience of your Desktop. Run SQL commands on your SQLCE databases, and view the results instantly.”

Name: RemoteSqlCe
Link: http://www.gui-innovations.com/html/remotesqlce.html
SQL CE Versions: 2.0, 3.0
Last updated: 2007
Platforms: Windows CE, Win32
”SqlServerCE is a great database for Pocket PC’s, but designing, maintaining and querying databases on the PPC, especially whilst developing, can be wearing. We developed this tool to help us when developing for SqlServerCE. With RemoteSQLCe you can completely manage your SqlServerCE database from the desktop. RemoteSQLCE works in the cradle, and over ethernet/bluetooth.” 

SQL Server Compact 3.5 SP2 now supports Merge Replication with SQL Server 2012

A major update to SQL Server Compact 3.5 SP2 has just been released, version 3.5.8088.0, disguised as a “Cumulative Update Package”. Microsoft knowledgebase article 2628887 describes the update.

The update contains updated Server Tools, and updated desktop and device runtimes, all updated to support Merge Replication with the next version of SQL Server, SQL Server 2012.

For a complete list of Cumulative Updates released for SQL Server Compact 3.5 SP2, see my blog post here.

It is nice to see that the 3.5 SP2 product, with it’s full range of device support and synchronization technologies is kept alive and kicking.

NOTE: I blogged about this update earlier, but it was pulled back. Now it is finally available, and all downloads can be requested. (I have downloaded all the ENU ones, anyway)

SQL Server Compact Toolbox 2.5–Visual Guide of new features

After more than 66.000 downloads, version 2.5 of my SQL Server Compact Toolbox extension for Visual Studio 2010 is now available for download. This blog post is a visual guide to the new features included in this release, many suggested by users of the tool via the CodePlex issue tracker

Properties of selected Database and Table displayed in Properties window

When you navigate the list of SQL Server Compact databases and database objects, the Toolbox now displays properties for a Database and a Table.

The Database properties are the following:

And the table properties are these:

DGML file (database graph) now also contains object descriptions

The database graph (DGML) file has been enhanced to display descriptions of Database, Table and Column objects, based on the documentation feature introduced in version 2.3:

Entity Data Model dialog now allows adding configuration for Private Desktop Deployment to app.config

When using Entity Framework with Private Deployment of SQL Server Compact, some entries in app.config are required (for Desktop application), as described here and here. This is required, as Entity Framework build in the DbProvider interfaces, which requires some configuration entries.

These settings can now be added when creating an EDM as shown below:

Ability to add 3.5 Connections from Toolbox

It is now possible to add 3.5 Database Connections to the Server Explorer and the Toolbox directly from the Toolbox, rather than having to go to Server Explorer, add the connection, and then Refresh the Toolbox. You can now do this without leaving the Toolbox, and the Toolbox will refresh “automagically”.

Improved VS UI Guidelines compliance

The Toolbars, SQL Editor Font, Dialogs (frame, background and buttons) have been overhauled to comply better with my recent discovery of “Visual Studio UI Guidelines”, available for download here. In addition, the Toolbox now follows the chosen Visual Studio Theme (to some extent, anyway).

This is the “new look” for the Toolbox Explorer and SQL Editor:

Other minor improvements and fixes

- Explorer Treeview: ArgumentException when getting connections
- WinPhone DataContext: Split files failed with empty database
- SQL editor: Check if .sqlplan is supported
- SQL editor: Save button was not working
- SQL editor: Results pane not always cleared
- SQL editor: Results as text improved formatting
- SQL editor: Text scrollbar was overlaid by splitter bar

As usual, the full source code for these new features is available on CodePlex for you to re-use or maybe improve!