Today, converted 2 heap tables to clustered ones, and halved the database size. Heap tables get so fragmented that they will have a gigantic amount of unused space. Here is a link to a great script to find out which tables are using up your disk space in SQL server – if the “Unused” space is abnormally large, you may have a fragmentation problem, and one of the best thing you can do is get rid of heap tables, by creating a clustered index or a primary key:

ALTER TABLE Customer ADD PRIMARY KEY (SID);

or

CREATE CLUSTERED INDEX IX_TABLE ON TABLE(COLUMN)


There are a few other ways to defragment tables – but they may not work well (or at all) on heap tables.

PS: If the above script is gone, google “Bill Graziano BigTables”.

Boo, hiss, popups! Web developers always hate popups. We hate them so much that we go to great length to fake them inside of the browser. And there are a lot of reasons for hating them, but there are also a few good use cases for them.

You can’t open a popup from the server side since it is, well, on the server; and you can’t use a <script> tag inside of a smart part, because it is rendered on an UpdatePanel, but you can still push out some script to be executed by the page via the ScriptManager control. Here is the code I use to navigate to a new web page via a popup:

ScriptManager.RegisterClientScriptBlock(this.Page, GetType(), "NavigateToUri",
    "window.open('" + uri + "', '_new', 'height:200,width:300,statusbar,menubar,resizable,titlebar,scrollbars');",
    true);

Note that if the URI comes from an external source (like, the database) you may want to sanitize it to avoid XSS attacks.

To be honest I don’t really like VBScript. Still, on Windows it is still the most conveniently available scripting engine and for many small scripts I don’t feel justified to drop Python on the customer’s machine. One of the most nagging issue that I had been living with so far is the need to copy a big cruft of utility function and constants from script to script, and last week I finally looked for (and found) a way to work around that.

In Windows 2000 and above you can use instead of the good old “vbs” file a new file format with the “wsf” extension. Basically this is a VBScript with some additional meta information, in XML form. This lets you do a few interesting things:

• Declare included script (number 1 advantage for me!)
• Reference constants from external libraries (this is pretty sweet too! No more “Const adUseClient = 3″ pasted in every script)
• Handle command line parameter (so you can do /contactid:C89983292 instead of remembering that the first parameter must be a contact id, and it can also generate a “Usage” blurb automatically)

To use them it is very easy: rename your file to .wsf, add the few required tags at the top of the file (and close them at the bottom), and don’t forget to enclose your script itself within a “CDATA” section otherwise characters like & or > will mess it up:

<?xml version="1.0"?>

<job id="UpdateContactOwner">
<runtime>
<description>
Updated the given contact seccodeid and account manager.
</description>
<named
  name="contactid"
  helpstring="Contact to be updated"
  type="string"
  required="true"/>
</runtime>

<reference object="ADODB.Recordset"/>

<script language="vbscript" src="UpdateContactOwner.vbs">
</script>
<script language="vbscript">
<![CDATA[
  If Len(WScript.Arguments.Named.Item("contactid")) = 12 Then
    UpdateContactOwner WScript.Arguments.Named("contactid")
  Else
    WSCript.Arguments.ShowUsage
  End If
]]>
</script>
</job>

Alright, this isn’t much, and is pretty old news, but it was pretty aggravating to look for it this week-end so I might as well jot it down for later.

Somehow last week I started getting some “Invalid Character” errors all over the place (or maybe I just started noticing them, I don’t know). Some came up in msbuild scripts, and some in Javascript files. They just looked like 1 or 3 gibberish characters at the very beginning of the file. I kind of dismissed it because the errors went away after I opened the file in vim and re-saved it, but they came back with a vengeance last week-end when I found out that was the ultimate cause for my Django templates messing up. I really found out I had a bona fide Django bug there for a moment, but it was just copying the “BOM” (byte order mark) from my source files.

So what is this BOM anyway? Simply put they are a few bytes inserted at the beginning of a (unicode) file to help the computer determine how to read it.
It doesn’t really make sense for UTF-8 (the most common default encoding) because the order in those is fixed! But for a double-byte encoding like UTF-16 you have to know whether to put the first byte first or last (there is a longer story to this “endianness” as they call it but let’s cut short here). ANYWAY, some programs will still put a BOM in UTF-8 file, consisting of the 3 bytes 0xEF, 0xBB, 0xBF. And some programs will manage to choke on it. OR, it will have some strange effect when you try to do some things to the files like concatenate them or whatever.

Where do they come from? It seems like Notepad (not Notepad2) will insert them automatically depending on your encoding settings.

To get rid of them, use “:set nobomb” in vim and save, or in Notepad2 change the encoding to “UTF-8″ (instead of “UTF-8 with signature”).

Here is the Wikipedia article which explains this in much more detail.

10 years ago I had to reboot from Linux (Slackware) to Windows 98 in order to use my printer.
5 years ago I had to reboot from Linux (Debian) to Windows XP in order to play games.
Now? I have to reboot from Windows Vista to Linux (Ubuntu) in order to use my scanner.

I am not bitter, really – just amused! And I can’t wait for Vista SP1 to (well, hopefully) bring back some of the usability.

Update: Yes after installing SP1 my #1 problem (the horribly slow file transfers) has been resolved, so I am now back on Vista.

The Saleslogix web client comes packaged with a wonderful logging system called log4net. This enables us to receive very detailed information about what is going on in every request, how the data is being read from the database, what business rules or events are being executed, etc. Unfortunately it is all disabled in the default installation. While I hope this changes in the near future to a configuration option, here is how you can turn it on for now. Warning: there is a significant performance impact to logging the debugging information – make sure you tweak this before deploying on production (you can restrict it to log only warning or errors).

• Edit the web.config file, you need to add a line in the top (after the <configSections> tag) that reads:
  

  <section name="log4net"
    type="log4net.Config.Log4NetConfigurationSectionHandler,log4net,
      Version=1.2.10.0, Culture=neutral, PublicKeyToken=1b44e1d426115821" />

• At the end of your web.config file (right before the closing </configuration> tag) you will need to paste the log4net configuration proper. I won’t lie, this is a pretty decent chunk of XML and I usually just carry the same one around. But in truth it is pretty simple: we are defining Appenders which is where you want to send the log output (eg to files, or to the event viewers), Loggers which are settings for where the log comes from (for example you can specify that you only want log messages of a certain severity for a given logger, or decide to send messages from some loggers to emails and not others), and finally a root setting which is simply the default settings for all loggers. These are the settings that I use for development – they configure the logger to show only warnings from NHibernate (you can change that to DEBUG but be warned that there is a lot of info and it does have a significant effect on performance at that level), and all messages from any other part of the system. Everything is sent to a text file under the Log subdirectory:

  <!-- This section contains the log4net configuration settings -->
  <log4net debug="false">
    <appender name="rollingFile" type="log4net.Appender.RollingFileAppender" >
  
      <param name="File" value="Log/log.txt" />
      <param name="AppendToFile" value="false" />
      <param name="RollingStyle" value="Date" />
      <param name="DatePattern" value="yyyy.MM.dd" />
      <param name="StaticLogFileName" value="true" />
  
      <layout type="log4net.Layout.PatternLayout">
        <param name="ConversionPattern" value="%d [%t] %-5p %c - %m%n" />
      </layout>
    </appender>
  
    <!-- Setup the root category, add the appenders and set the
       default priority -->
  
    <root>
      <priority value="DEBUG" />
      <appender-ref ref="rollingFile"/>
    </root>
  
    <logger name="NHibernate">
      <level value="WARN" />
    </logger>
  
    <logger name="NHibernate.SQL">
      <level value="ALL" />
    </logger>
  </log4net>

• Create a Log directory under the folder and set the permission to be modifiable by Everyone – this is where log4net will send the output.
• You are almost done, the last thing is you need to create a file called Global.asax in the root of the web client with the following code to initialize the logging system:
  

  <%@ Application Language="C#" %>
  
  <script runat="server">
  
      private static log4net.ILog _log;
  
      /// <summary>
      /// Initialize web leads library.
      /// </summary>
      /// <param name="sender"></param>
      /// <param name="e"></param>   
      void Application_Start(object sender, EventArgs e)
      {
          log4net.Config.XmlConfigurator.Configure();
          _log = log4net.LogManager.GetLogger(typeof(global_asax));
          _log.Info("Application Started");
      }
  
      void Application_End(object sender, EventArgs e)
      {
          //  Code that runs on application shutdown
          _log.Info("Application Ended");
          log4net.LogManager.Shutdown();
      }
  
      /// <summary>
      /// Redirect to generic error page.
      /// </summary>
      /// <param name="sender"></param>
      /// <param name="e"></param>    
      void Application_Error(object sender, EventArgs e)
      {
      }
  
      void Session_Start(object sender, EventArgs e)
      {
          // Code that runs when a new session is started
  
      }
  
      void Session_End(object sender, EventArgs e)
      {
          // Code that runs when a session ends. 
          // Note: The Session_End event is raised only when the sessionstate mode
          // is set to InProc in the Web.config file. If session mode is set to StateServer 
          // or SQLServer, the event is not raised.
  
      }
  
  </script>

That’s it! Browse to the client and make sure the “log.txt” file gets created.

There is one more thing I like to do, add this code in the Page_Load of SmartParts/General/GeneralException.ascx, it will give you a traceback in the log whenever you hit an exception (the ones that are reported with the “Saleslogix has encountered an error” screen, not the ones that give you a Yellow Page Of Death):

if (Page.Request.QueryString["exceptionid"] != null)
{
  Exception ex = Sage.Platform.Application.ApplicationContext.Current.State[
     Page.Request.QueryString["exceptionid"]] as Exception;
  if (ex != null)
  {
    ExceptionID.Text = ex.Message;
    log4net.LogManager.GetLogger(typeof(GeneralException)).Warn("Exception Reported", ex);
  }
}

Now in your code, if you want to output some log messages, put code similar to this at the top of your class to declare a logger:

private static readonly log4net.ILog LOG =
  log4net.LogManager.GetLogger(typeof(myclass));

and in your code add calls to LOG.Info, LOG.Debug, etc.

See the log4net home page for more information about this excellent tool.

Kidding! But, now that Microsoft has opened the trail, one can dream? Imagine how much easier it would be to figure out this stuff if you could actually step through it instead of having to work with the dead code in Reflector.

One of the new possibilities I was pretty excited about with the 7.2 web client is that we can finally start using our standard source control tools (we use Subversion – very simple yet powerful) to manage revisions. It is still not perfect because there are a few tidbits that are hard to get out of the database but in practice I have not found that to be much of a problem (UPDATE – see below). What source control gives us is a nice, easy to use way of:

• tracking changes between revisions (i.e. what the heck did I break between last Wednesday and today). Also makes it a bit easier to apply the Sage bundle since I can apply the bundle and then use the source control to tell me what changed.
• archiving your source code (other than backing up the whole database, which can be a bit of a hassle to restore).
• collaborating on changes between users (i.e. if I am messing with something and Joe is messing with an unrelated feature, we don’t have to worry about temporarily breaking the website, since we are working on separate copies – though obviously we have to watch out when merging the changes back together).
• it also lets you have several projects against the same database (eg for experimenting with a new feature without having to set up a new database). And when you are done experimenting, you can toss it out, or automatically merge into the production copy. Anyone who has ever used the Bundler will recognize how wonderful that is!

Here is what I had to do to make it happen, like I said it was pretty easy once I found where everything was:

• In your “Project Workspace Manager” right click and select “Add”:
  
• Enter a meaningful name: the project workspace manager does not filter the list so you will have to be able to recognize it among all the other projects you might be working on!
• Enter the path to export, make sure it is on the local drive and not too long: some of the files in AA are pretty long and combined with something like C:\Documents And Settings\domain.user\Documents\Visual Studio 2008 Projects… well it gets past the limit pretty quickly as I found out first-hand.
• And finally, make sure you check the box to “Export Files Upon Creation”, and click “Create”:
  
• Now double click on that shiny new project to open it. Any change you make to the entity model or the quick forms will now be saved to the specified folder instead of into the database. There are still a few things pointed to the database, for example, the Sage.Entity.Interfaces.dll that is generated when you build the project, and the support files (which includes all the assemblies used by the web site). It is possible to get the support files out but not really worth the hassle – first of all it is a pretty big directory, a lot of it is binary files that will just end up cluttering your repository for no practical purpose, and secondly I prefer doing my changes by copying the files to a separate directory and modifying them there, this way I can see exactly what I am working on (I have a build script that automatically deploys them from there to the web site, but that is another story).
• You can now browse to the folder and import it into your repository using whatever is appropriate… for example with TortoiseSVN I just right click and import.

That’s it! One more thing I have to say if you are not familiar with source control is that it is even more fantastic when it is used with actual source code (as opposed to the XML form used by AA which is pretty verbose and automatically generated thus a bit harder to use in comparisons). This is where it really shines enabling you to see who did what and when and decide how you want to combine it.

UPDATE (2008-01-19): In light of a recent misadventure where I accidentally overwrote the database-stored VFS I would indeed recommend storing the support files into the source control as well. There is about 26 megs of them, and most of it is binary file, and they are not going to change often, but you know what, I’d rather know where they are. To do that, expand the portal (”Sage Saleslogix” or “Sage Saleslogix Customer Portal”), right click on the “Support Files” item and select Copy File. Select an empty target directory. Next, right click, select “Set Source Path”, and point it to your new directory. Check it into source control. There is STILL going to be some stuff saved into the VFS (eg the Sage.Entity.Interfaces file), but we’ll have to let it be for now. And the good side is, the base is going to be the same on all databases, so what I do is store the “Vanilla” 7.2.1 files in a specific path and have all databases reference to that same path. Then, I store the “delta” in a separate folder. I am considering doing that with the model too now (by storing it as a branch in subversion) but not quite there yet.

UPDATE (2008-03-17): On Saleslogix 7.2.2 you may get an error when compiling the platform, something that looks like “Unable to read snippet file, ‘\Entity Model\SalesLogix Application Entities\LeadAddress\.svn\text-base\Sage.SnippetLibrary.CSharp.@.07ffd774-6142-4b8a-a48a-09616effccea.codesnippet.cs.svn-base’. Unknown extension. Supported extensions are: .cs, .vb“. Sage tries to compile every file that contains “codesnippet” in its name. A small update to the Sage.Platform.dll is necessary to get past this – I may write more fully on the subject of how to patch the Sage libraries later but you can download the patched dll here (copy the dll to your C:\Program Files\Saleslogix\Architect\Platform folder) or the patched source code here (compile it with ilasm.exe).

Here are a few new year resolutions (on the technical side, anyway), just so I can laugh at them next year:

• Learn Django – I need a breath of fresh air from the ASP.NET sometimes, I settled on Django after hesitating between that ASP.NET MVC (too close to home… I prefer something not running on .NET for a change), Ruby on Rails (bad experience with Ruby in the past… don’t ask), and a few more esoteric ones like Seaside or YAWS
• Learn F#, or maybe Haskell? Something that is different enough from C# to give me a different perspective. I am worried that F# is probably still very close since it runs on the CLR but it might be hard to find a practical use for something like Haskell or Lisp
• Learn more about the CLR – I picked up the book CLR via C# and intend to give it a thorough reading

Obviously SLX 7.2, ASP.NET and MS Ajax will have a place there too but that is pretty much a given even without a resolution!

On the non-technical side I would like to resolve NOT to adopt another dog this year but that might be a tough one (gee I hope not, 4 is more than enough!!)