Friday, June 20, 2008

Some of my posts that I get the most recurring email/inquiries on are my various posts about the STDF (Standard Test Datalog Format) parser that I originally created as an experiment back in 2005.  After some help from a former collegue, I am pleased to announce that this is finally available on CodePlex as LinqToStdf!

It is a managed library for processing STDF files, and gives you a model to explore the data via Linq queries.  This means, you can leverage the wide variety of managed languages (C#, VB, F#, JScript, IronPython, IronRuby, Managed C++, Boo, etc.) to process the data in STDF files.  It also works in Silverlight!  It has built-in support for the V4 spec, but it's highly extensible and should be able to parse any version of the file format as well as custom records.  It can be configured to be highly strict and throw on format errors, or be robust in the face of issues that normal STDF processors would choke on to the point of being able to detect and repair corruption on the fly.

If that interests you, I'd love for you to drop by and take a deeper look at it and get involved in its ongoing development.  I've already got at least one person interested enough to contribute and ensure its success as a community project.  There is currently a "beta" release available, and hopefully we'll whip it into shape enough to call it v1.0 soon.

My hope is that this can be an adoption driver for .NET in the semiconductor industry and that through this project I can be an ambassador for the CLR in that area.

posted on Friday, June 20, 2008 1:05:13 PM (Pacific Standard Time, UTC-08:00)  #    Comments [2]
 Tuesday, June 10, 2008

I've been playing alot with beta 2 of Silverlight 2, and I've been totally amazed at the scenarios it enables.  Working hard down inside the CLR engine, we're sometimes insulated from some of the innovation going on higher in the stack and it blows us away when we do get a chance to see it.

One of the very cool things in Silverlight is "Deep Zoom", which came from the SeaDragon project from Microsoft Research.  I decided to try it out myself on a very large panorama that I made a long time ago in New Orleans.  Unfortunately, the current toolset seems to trip over the large file size (20516x15291).  I'm trying to find out the real story behind the limitation.  All I know right now is that smaller files work.

So, I decided to try to slice it up into smaller, manageable chunks and just butt them against one another to simulate one large file.  The problem was that I couldn't find a tool to do this that didn't also trip up over the size of the file.  So, I wrote my own.  There are likely better ways to do this, this was just a quick and dirty attempt to make something that didn't totally crawl to a halt due to page faulting (or outright throw OutOfMemoryException).

The code below is what I came up with.  I use System.Drawing.Bitmap, lock the portions of the image I need, and do the data copying myself.  I ended up with this solution because GDI+ (DrawImage) seems to like to make alot of buffers (big ones in this case), and I couldn't fall back on good ol' bitblt because I couldn't get the right kind of data structures without more copying of the data.  This runs plenty fast and only takes up marginally more memory than it takes to represent the original and the size of one of the destination tiles.

Unfortunately, The MultiScaleImage does some weirdness with image that are butted up against each other, and you get as much as a whole pixel of "space" between them, depending on your zoom level. I'm still looking into other possible workarounds. So, it's wasn't ultimately useful, but I thought the code was interesting enough, and probably has academic usefulness.  So, enough talk, here's the code:

   1:  using System;
   2:  using System.Collections.Generic;
   3:  using System.Drawing;
   4:  using System.Drawing.Imaging;
   5:  using System.IO;
   6:  using System.Linq;
   7:  using System.Runtime.InteropServices;
   8:   
   9:  namespace ImageSlicer {
  10:      static class Extensions {
  11:          public static IEnumerable<int> Times(this int number) {
  12:              for (var i = 0; i < number; i++) yield return i;
  13:          }
  14:      }
  15:   
  16:      class Program {
  17:          [DllImport("msvcrt.dll", SetLastError = false)]
  18:          static unsafe extern byte* memcpy(byte* dest, byte* src, int count);
  19:   
  20:          static void Main(string[] args) {
  21:              var sourcePath = args.FirstOrDefault();
  22:              if (String.IsNullOrEmpty(sourcePath)) {
  23:                  Console.WriteLine("No source image path provided");
  24:                  ShowUsage();
  25:                  return;
  26:              }
  27:              if (!File.Exists(sourcePath)) {
  28:                  Console.WriteLine("Source image path doesn't exist");
  29:                  return;
  30:              }
  31:              var gridSizeStr = args.Skip(1).FirstOrDefault();
  32:              int gridSize = 4;
  33:              if (gridSizeStr != null && !int.TryParse(gridSizeStr, out gridSize)) {
  34:                  Console.WriteLine("Could not convert {0} to a valid grid size",
  35:                      gridSizeStr);
  36:                  return;
  37:              }
  38:              if (gridSize < 2) {
  39:                  Console.WriteLine("The grid size must be greater than 1.");
  40:                  return;
  41:              }
  42:              try {
  43:                  Console.WriteLine("Slicing {0} into a {1}x{1} grid.",
  44:                      Path.GetFileName(sourcePath), gridSize);
  45:                  Console.WriteLine("Loading...");
  46:                  using (var sourceBitmap = new Bitmap(sourcePath)) {
  47:                      Console.WriteLine("Source Image: {0}x{1}",
  48:                          sourceBitmap.Width, sourceBitmap.Height);
  49:                      var sliceWidth = sourceBitmap.Width / gridSize;
  50:                      var sliceHeight = sourceBitmap.Height / gridSize;
  51:                      Console.WriteLine("Each slice: {0}x{1}", sliceWidth, sliceHeight);
  52:                      int tile = 0;
  53:                      foreach (var row in gridSize.Times()) {
  54:                          foreach (var column in gridSize.Times()) {
  55:                              Console.WriteLine("Creating {0} of {1} ({2},{3})",
  56:                                  ++tile, gridSize * gridSize, column, row);
  57:                              using (var destBitmap = new Bitmap(sliceWidth, sliceHeight)) {
  58:                                  var sourceData = sourceBitmap.LockBits(
  59:                                      new Rectangle(column * sliceWidth,
  60:                                          row * sliceHeight, sliceWidth,
  61:                                          sliceHeight),
  62:                                      ImageLockMode.ReadOnly, PixelFormat.Format24bppRgb);
  63:                                  var destData = destBitmap.LockBits(
  64:                                      new Rectangle(0, 0, sliceWidth, sliceHeight),
  65:                                      ImageLockMode.WriteOnly, sourceData.PixelFormat);
  66:                                  unsafe {
  67:                                      byte* pSrc = (byte*)sourceData.Scan0.ToPointer();
  68:                                      byte* pDest = (byte*)destData.Scan0.ToPointer();
  69:                                      foreach (var line in sliceHeight.Times()) {
  70:                                          memcpy(pDest, pSrc, sliceWidth * 3);
  71:                                          pSrc += sourceData.Stride;
  72:                                          pDest += destData.Stride;
  73:                                      }
  74:                                  }
  75:                                  sourceBitmap.UnlockBits(sourceData);
  76:                                  destBitmap.UnlockBits(destData);
  77:                                  destBitmap.Save(String.Format("{0}_{1}_{2}.png",
  78:                                      Path.GetFileNameWithoutExtension(sourcePath),
  79:                                      column, row), ImageFormat.Png);
  80:                              }
  81:                          }
  82:                      }
  83:                  }
  84:              }
  85:              catch (Exception ex) {
  86:                  Console.WriteLine("Error processing image.");
  87:                  Console.WriteLine(ex.ToString());
  88:              }
  89:          }
  90:   
  91:          private static void ShowUsage() {
  92:              Console.WriteLine("Usage:");
  93:              Console.WriteLine("ImageSlicer.exe sourceImage [gridSize]");
  94:          }
  95:      }
  96:  }
posted on Tuesday, June 10, 2008 10:01:35 PM (Pacific Standard Time, UTC-08:00)  #    Comments [3]