Tuesday, October 04, 2005
I've had previous problems with obfuscating code with Dotfuscator.  I seem to be cursed...I've found another one.    Is no one else using this product to obfuscate CLR 2.0 code?  This one is quite wicked.  I spend several hours digging through before and after IL. (which is difficult when the purpose for obfuscation is to make it difficult to read) Here's the recipe for disaster:
  • A generic method with the following characteristics:
    • More than one type parameter.
    • A return type composed of one or more of the method's type parameters other than the first one.

Some C# example signatures (Not useful or practical, just simple examples that show the problem):

static IEnumerable<TValues> FindDictionaryValues<TKey, TValue>(IDictionary<TKey, TValue> dictionary)

static TValue FindAValueInADictionary<TKey, TValue>(IDictionary<TKey, TValue> dictionary)
 
After obfuscation with dotfuscator (even with all obfuscation options disabled), those two signatures will look like:

static IEnumerable<TKey> FindDictionaryValues<TKey, TValue>(IDictionary<TKey, TValue> dictionary)

static TKey FindAValueInADictionary<TKey, TValue>(IDictionary<TKey, TValue> dictionary)
 
What happens is fairly simple.  Any reference to the method's type parameters in the return type becomes a reference to the first type parameter (usually !!0 in the IL).  What's more, this corruption also happens at the call site, which I didn't discover until I had written a regular expression to find and repair the corrupt signatures.
 
It is quite as if generics support was cobbled on as a hurried afterthought/sellingpoint rather than being properly integrated into their codebase.  Their support for 2.0 is poor, even for "beta" status.  However, Preemptive seems interested in fixing the issues.  I'll update when I know more.
 
[UPDATE 10/06/2005] I wanted to let everyone know that PreEmptive is being very responsive to this issue.  They've delivered a patch and I am currently evaluating it.  So far, it appears to fix my issues, but we're working through a few other issues.  Will update when I have more info.
posted on Tuesday, October 04, 2005 2:08:26 PM (Pacific Standard Time, UTC-08:00)  #    Comments [0]
 Wednesday, September 14, 2005

I've been distracted all morning, drooling over the C# 3.0 features.  First, I was upset that they would tease us with 3.0 features before 2.0 is even released yet (and it's still a little silly), but being able to download, play with them, and run them makes a world of difference.  Here's my favorites distilled for those who don't want to check it out.

  • Tons of type inference stuff.
    • List<int> list = new List<int>();
    • becomes
    • var list = new List<int>();
    • I'm not sure how I feel about "var".  It makes it feel a little like javascript.  I can see how that makes the language less ambiguous, and three letters isn't that bad.
  • Tons of initialization stuff
    • Initialize any collection-type-thing like you would an array. (new List<int> {1,2,3,4})
    • Intialize objects with a named-parameter feel (new Person {FirstName="Mark", LastName="Miller} //not a constructor)
  • A few AOP-ish goodies with extension methods - This is extremely cool.  Add methods to string, or anything else.  Very sweet.
  • "Fix" the anonymous delegate syntax with lambda expressions
    • delegate(int x) {return x+1;}
    • becomes
    • x => x + 1
    • This will be hard to approach for some, but I think it's great
  • Anonymous Types - This is excellent for working with data where you don't really need a formal type, you just need to work with some data.
  • Language Integrated Query - This is the grand-daddy of them all.  Query against objects, query against XML, query against relational data, query against anything!  All using first-class features of the language!  I'm pretty excited about this.

I'm dissapointed in the lack of first-class duck typing support (late binding).  Yeah, we get half-way there through the use of query and anonymous classes, but there's some scenarios where that seems like hammering a tack with a sledgehammer.

I'm really excited in the innovation I see in the CLR-world.  Sure, lots of the above represent old ideas, but the query stuff represents big innovation in that space that fits right in where the "hurt" is.

posted on Wednesday, September 14, 2005 9:17:46 AM (Pacific Standard Time, UTC-08:00)  #    Comments [1]
 Saturday, September 03, 2005

Doesn't that sounds like an awesome band name?  Someone here just said that and I had to write it down.  This seemed like as good a place as any.

posted on Saturday, September 03, 2005 1:50:13 PM (Pacific Standard Time, UTC-08:00)  #    Comments [3]
 Thursday, September 01, 2005

Most of my day today was tracking down a problem with our app when we obfuscated it.  We were getting a System.EntryPointNotFoundException in the obfuscated version, while the normal one worked fine. The stack trace contained the following on the top of the stack trace:

at System.Collections.Generic.IEnumerable`1.GetEnumerator() +0

No concrete class, and an assembly code offset of 0. Weird, eh? After tweaking with lots of settings and digging through the IL of several iterations of differing obfuscation techniques, I finally determined the problem.

I had three overloads of a method that each took a different parametrization of IEnumerable<T>.  For the sake of example, we'll say IEnumerable<A>, IEnumerable<B>, and IEnumerable<C>.  In the obfuscation process, calls to any of the three were being changed to the IEnumerable<A> overload.  Sure enough, PEVerify was giving errors about the wrong type being on the stack. (I was going to show the IL before and after, but I just realized that I deleted the non-working one after fixing it. I'll post it if I can find it again)  It seems that Dotfuscator was not distinguishing properly between overloads if they only differ by parameterization types. I have not verified it, but this may have been aggravated by iterator methods.

We're working up some sample code to send to Preemptive so they can address the problem, but debugging it was frustrating enough to post a synopsis of the account for Google to find.

posted on Thursday, September 01, 2005 12:13:07 PM (Pacific Standard Time, UTC-08:00)  #    Comments [1]
 Monday, August 22, 2005

A while back, I showed you my file-based viewstate persistence solution.  Thanks to Google search hits and traffic from Scott Hanselman's analysis, it's been one of my more popular entries.  With ASP.net's improvements in this area, I felt that it was due for an update.  So, here's a quick whack at it.  The usual disclaimers apply.

2.0 adds the notion of "control state" to state persistence, which is very cool.  It's an opt-in mechanism for things that need to survive postbacks even if ViewState is turned off.  In addition, there's some new flexibility with page adapters and such, but we'll ignore that complexity for now and go for a direct port of my old sample, but include the new ControlState mechanism.

public class FileBasedPageStatePersister : HiddenFieldPageStatePersister {

      public FileBasedPageStatePersister(Page page) : base(page) {}

 

      public override void Load() {

            //let the base class do its thing

            base.Load();

            //get the control state

            object baseControlState = base.ControlState;

            if (baseControlState != null) {

                  //the control state should be our Guid

                  Guid guid = (Guid)baseControlState;

                  //read the contents of the file and set the two states

                  using (TextReader reader = new StreamReader(CreateOfflineViewStateFilePath(guid))) {

                        Pair pair = this.StateFormatter.Deserialize(reader.ReadToEnd()) as Pair;

                        base.ViewState = pair.First;

                        base.ControlState = pair.Second;

                  }

            }

      }

 

      public override void Save() {

            //create a guid for this viewstate

            Guid guid = Guid.NewGuid();

 

            //serialize the states into a temp file

            using (TextWriter writer = new StreamWriter(CreateOfflineViewStateFilePath(guid))) {

                  Pair pair = new Pair(base.ViewState, base.ControlState);

                  writer.Write(this.StateFormatter.Serialize(pair));

            }

            //trick the normal system into thinking all it needs to save is the guid

            base.ControlState = guid;

            base.ViewState = null;

            base.Save();

      }

 

      string CreateOfflineViewStateFilePath(Guid guid) {

            //TODO: put these files whereever you like

            return Path.Combine(Path.GetTempPath(), string.Format("{0}.viewstate", guid));

      }

 

}

So, we immediately see that it's much shorter.  This is because ASP.net uses a mechanism very similar to my 1.1 solution, so alot of the plumbing is built-in. The only thing you need to do to use it is override the PageStatePersister property on Page and return one of these. Again, we're piggybacking on the "hidden field" persistence mechanism and using that to store our Guid for the request.  Not much different, and I'm pretty happy to say that converting to this model from my old is very simple.

Another interesting idea would be to leave the ControlState in the hidden field, and only store the ViewState in the file. it would be a simple change that I'll leave as an exercise to you.  Then, you could be very aggresive about purging old or large files without worrying about breaking anything (provided of course that you've made your controls tolerant to such a method).

posted on Monday, August 22, 2005 1:21:14 PM (Pacific Standard Time, UTC-08:00)  #    Comments [0]
 Saturday, August 20, 2005

My friend Peter and I often use Remote Assistance to collaborate on projects.  He and I were doing some things today in Visual Studio, and while I was watching, a beautiful window appeared that showed the open files and windows and such, complete with appropriate icons. "Whoa!", I said, "What was that?!"  "What", he replied, "this?"  The window re-appeared.  Turns out, it was the Ctrl-Tab window.  For some reason, I never even thought of using it to browse the open windows. I always assumed I had to navigate the frightful tabbed interface, which just might actually be usable with that little trick.

I felt better when I typed in a type name and Intellisense didn't recongnize it.  A quick right-click->Resolve instantly added the appropriate using statement.  Peter hadn't seen that feature.  Although I guess Ctrl-Tab's been a standard for much longer than Resolve.

posted on Saturday, August 20, 2005 11:26:33 AM (Pacific Standard Time, UTC-08:00)  #    Comments [0]
 Thursday, August 18, 2005

This past week, I've found myself writing alot of methods with the following pattern:

IEnumerable<T> DoSomething(IEnumerable<T> stream);

Then I use C#'s wonderful new iterator syntax to manipulate the stream, or simply intercept and process the data.  I've found this wonderfully useful for alot of the statistical type algorithms that I implement for our project.  It's been especially cool to combine this with a bit of reflection for creating "dynamic filters" based on user input.  All of a sudden, all your algorithms are incredibly flexible with very little effort.

The beauty of this pattern is that it can be chained.  Obviously, there is a practical limit here, but I've found it extremely useful and I thought I'd share it for those who haven't stumbled across this concept before.

Now, if we could only create iterators using anonymous delegates, then I would proceed to drown in my own drool.

[UPDATE] fixed title grammar. Sorry if that pops it back up in your aggregators, RSS readers.

posted on Thursday, August 18, 2005 12:29:34 PM (Pacific Standard Time, UTC-08:00)  #    Comments [0]