A large part of my time at work recently has been addressing scalability issues in some applications. Most of these stem from poor use of memory. Most of these applications were proof of concept that made their way into production code for whatever reason. More and more we identify problems and when we identify them to the developer, this is the typical exchange:
Me: You're taking too much memory here by using type Xxxxx when you could have gotten by with type Yyyyy Dev: I have to use type Xxxxx, that's what the data is.
Me: You're taking too much memory here by using type Xxxxx when you could have gotten by with type Yyyyy
Dev: I have to use type Xxxxx, that's what the data is.
Sometimes, you just have to take a step back and look at your bytes. Sometimes, you need a profiler, but sometimes you can just do some math.
For example, if I have an array of dates that correspond to events, a logical choice for a datatype is System.DateTime. However, if you've got millions of events, System.DateTime is relatively expensive. Especially if you consider that your event times only have resolution down to the nearest minute and only represent times within the last several weeks. The range, or domain of DateTime (ticks since the epoch) is overkill for your circumstances. You may have been able to get away with UInt16, which would have been 2 bytes instead of 8, which is a huge reduction. (Provided that you've shown this array of DateTime to be a significant part of your memory consumption.)
I'm sure there is some official name for this, but I'll refer to it as "constrained domain", where you know something about your circumstances or usage that allows you to reduce the range or domain of a concept in order to store it as something smaller to improve memory consumption. Remember, just because a big, rich datatype makes things easier, matches a db schema, or is named the same as the concept you need to store, doesn't mean it's a good match for your scenario.
There are, of course, other considerations to make. The Y2k problems were the direct result of taking this concept to the extreme (of course resources were in much shorter supply then than they are today).
So, when you are looking for places to trim your memory usage, look for places where you can constrain the domain of a concept and store it in fewer bytes. Also, if you're in CLR-land when you do this, look for boxing value types. Remember, bytes in a System.Collections.ArrayList take up 5 bytes each, not 1. (9 bytes in 64-bit land) (Yeah generics)
Remember Me
Page rendered at Sunday, October 12, 2008 12:31:22 AM (Pacific Standard Time, UTC-08:00)
Disclaimer The opinions expressed herein are my own personal opinions and do not represent my employer's view in anyway.