Brothers In Code

...a serious misallocation of .net resources

Goodbye Nunit-agent.exe

Once I started using Visual Studio 2010, I could no longer attach to nunit to debug my tests.  Instead I found that I had to attach to nunit-agent.exe instead.  Up till now this was a non-issue.  However, some performance profilling methods require a start-up app, and nunit starting it's own process was confusing it so I was forced to find the reason for this new process.

It turns out it's because nunit by default is set to run under the 2.0 framework and it needs to run a separate process in order to test 4.0 assemblies.  The good news is that you can force nunit to run under 4.0 with this change to your nunit.exe.config:

<startup>
<requiredRuntime version=”v4.0.30319″ />
</startup>
 
You can verify the before and after effect by looking at Tools -> Test Assemblies in nunit.

Parallel.For - Multithreading Made Easy

I was using nunit to make sure a checkbits/signing algorithm was doing what it was supposed to do. Like everything in cryptography it was slow. The following loop took more than 16 seconds:


      for (int i = 0; i < 10; i++)
      {
        var value = (UInt64)rng.Next((Int32)cipher.MaximumSequence);
        ulong? lastHackedValue = null;
        //for all the checkbit combinations...
        for (int j = 1; j < Math.Pow(2, cipher.CheckBits) - 1; j++)
        {
          //move the checkbits over to the correct spot
          var signature = (UInt64)j << cipher.SequenceBits;
          var hackedValue = value + signature;
          try
          {
            cipher.Unsign(hackedValue);
            if (lastHackedValue == null)
              lastHackedValue = hackedValue;
            else
              Assert.Fail("Too many successful cracks for value {0}.  Hacked Value 1:{1}, Hacked Value 2:{2}", 
                i, lastHackedValue, hackedValue);
          }
          catch(ApplicationException ex)
          {
            Assert.That(ex.Message == "Invalid signature");
          }
        }
      }

I tried using the Parallel class – sending my cpu utilization from 12 to 100% and I got it down to 5 seconds:


        var parallelresult = Parallel.For(0, 10, i =>
        {
    //...same for loo
    });


I noticed that assertion failures were not being displayed in nunit correctly.  It turned out the .For call was wrapping them in an AggregateException so I wrapped the code like this:

    try
      {
        var parallelresult = Parallel.For(0, 10, i =>
        {
    ...
        });
      }
      catch(AggregateException ex)
      {
        if(ex.InnerException is AssertionException)
        {
          throw ex.InnerException;
        }
        else
        {
          throw ex;
        }
 
      }

The "Could Not Load File Or Assembly" Error Strikes in a New Way

I got bit by this old error in a new way the other day so I thought I should make a quick checklist for myself since I made some mistakes in my troubleshooting process. 

First I would start by reading How the Runtime Locates Assemblies.  However, unless you are depending one of the methods that change the normal process, like including additional "codebases" or publisher policy redirection, then we can widdle this down to a few simple causes:

  • A Referenced Assembly is not in the Application's Path
  • A Referenced Assembly is not in the Global Assembly Cache
  • The Referenced Assembly is a different version than the installed assembly.
  • The Application is 32 bit and the dll is 64 bit

The Old Problems - Local Files and the GAC

Many if not most applications rely on a simple xcopy deployment, meaning that all of the application's files will be bundled together in the same directory.  If you're simply copying or app folder from one place to another and it's not working on the new machine than either you simply missed a dll or you were using a GAC'd assembly on your development machine and didn't realize it.  If you're using a third party component that was installed with a setup program, than there's a good chance that your referenced assembly is in the GAC.  In that case you need to decide if you you want to run the third party install program as part of your deployment or figure out which dlls that you need and copy them to the folder.

GAC or otherwise, your app isn't going to find anything if it references a different version than what you have installed.  There are two options if realigning the deployed version isn't an option.  Either change "Specific Version" to false in the reference properties in Visual Studio or do a policy redirection.  I actually recommend the latter since big companies will include policies for the GAC that point old references to newer versions.  On the flip side, changing specific version to false will let your app load any version including an ancient one which might give you all sorts of strange errors.

 The New Problems - 32 Bit, 64 Bit and the Platform Target Configuration

It's the last cause that is the inspiration for this post since it has now bit me multiple times.  In my case i was referencing Oracle.DataAccess.dll from Oracle's data provider for .net.  ODP.net throws in some additional variables like PATH and "ORACLE_HOME" environment variable dependencies that threw me off correctly debugging the problem.  After double checking that the dll was in the GAC, and then even copying the dll to the app folder in desperation, I still was getting the error in question.  The error also included the phrase "or one of its dependencies" which kept doubling me back to a PATH problem.  In retrospect I really don't ever remember a case where this error was caused by a dependent dll so I'm not sure why I put so much stock in that route.

Finally I got my wits back and loaded a consistent savior, Sysinternals Process Monitor.  I don't know why I'm so slow to use this tool some times.  Maybe it's the 5 minutes you have to spend getting the filters right, but 9 times out of 10 it more or less tells me exactly what is wrong.  Sure enough I saw the program trying to find my dll in GAC_32.  Why is it looking for a 32 bit dll when we are on a 64 bit machine with the 64 bit oracle provider....you dumb-ass I thought to myself, knowing that I had seen this before.  Sure enough the task manager showed the infamous *32 next to my process.  I went back to visual studio and looked that the build properties for my exe project - once again Platform Target was set for "x86".  I set it to "Any CPU", recompiled and the stupid world was right again.

Getting Active Directory Information From ASP.net The Easy Way

A couple years ago I wrote a short article on an old blog about connecting to Active Directory from an ASP.net project.  I thought the article was fairly complete until I tried to help somebody else do the same thing.  I had forgotten two major parts:

  • Manually sifting thru the different AD containers to find the right LDAP string. 
  • Ensuring the correct security context.

The security context can be an easy thing.  A console or winforms .net app runs in the context of the currently logged on user.  For simplicity the ASP.Net development server in Visual Studio does the same thing.  So as long as your user account as access to the AD (which is the norm unless your AD admin has locked down querying), then you should be fine while you are developing.

Running under IIS is a different story.  Depending on the version, the user context of the iis process could be all sorts of things - ASPNET, Network Service, Local Service, the new IIS app pool accounts, etc.  During testing the easiest thing to do is enable impersonation:


<configuration>
  <system.web>
    <identity impersonate="true" userName="contoso\Jane" password="********" />
  </system.web>
</configuration>

Optionally, you can remove the userName and password attributes and make sure that NTLM/Integrated security is still checked in the IIS configuration.  This will make IIS operate similar to the ASP.Net Development Server and use the current user as the execution context.

So with the security context out of the way, the next task is to actually query AD for a user object.  In my old example, I would specify the container to connect to whe creating a DirectoryEntry object.  For example:


System.DirectoryServices.DirectoryEntry directoryEntry
        = new System.DirectoryServices.DirectoryEntry(LDAP://CN=Users,DC=contoso,DC=com);

This worked fine for me but was a bit to limiting as an example to go by.  Many domains have an additional "corp" domain context.  "Users" might be named something different like "DomainUsers".  There are all sorts of variables with AD.  To combat this I added an extra query to "RootDSE" which is sort of the current context.  With that I'm able to get the address of an actuall domain controller and from there, can make the query on a higher level.  In the example below, I'm trying to get the AD User object for the currently authenticated user, but there's lots of diagnostic output that would let you find other things or narrow down the container that you're querying (for performance reasons).

 


<%@ Page Language="C#" %>
<%@ Import Namespace="System.DirectoryServices" %>
<%@ Import Namespace="System.Collections.Generic" %>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<script runat="server">

</script>

<html xmlns="http://www.w3.org/1999/xhtml">
<head runat="server">
  <script runat="server">
    protected void Page_Load(object sender, EventArgs e)
    {
      OptLocalUser.Text = User.Identity.Name;
     
      Dictionary<String, String> directoryPropertyDictionary;
       
     
      //get the currently connected AD server info:
      DirectoryEntry de = new DirectoryEntry("LDAP://RootDSE");
     
      directoryPropertyDictionary = new Dictionary<string, string>(de.Properties.Count);
      foreach (string key in de.Properties.PropertyNames)
      {
        String allValues = "";
        foreach (object value in de.Properties[key])
          allValues += value;
        directoryPropertyDictionary.Add(key, allValues);
      }
      GridView1.DataSource = directoryPropertyDictionary;
      GridView1.DataBind();
     
      //connect to the current server
      de = new DirectoryEntry("LDAP://" + de.Properties["dnsHostName"][0].ToString());

      DirectorySearcher ds = new DirectorySearcher(de);
      String[] nameParts = User.Identity.Name.Split('\\');
      ds.Filter = "(&(objectClass=user)(sAMAccountName=" + nameParts[1] + "))";
      SearchResult result = ds.FindOne();


      OptUserPath.Text = result.Path;
      directoryPropertyDictionary.Clear();
      foreach (string key in result.Properties.PropertyNames)
      {
        String allValues = "";
        foreach (object value in de.Properties[key])
          allValues += value;
        directoryPropertyDictionary.Add(key, allValues);
      }
      GridView2.DataSource = directoryPropertyDictionary;
      GridView2.DataBind();

      OptLocalUser.Text = User.Identity.Name;
    }
  </script>
</head>
<body>
    <form id="form1" runat="server">
    <div>
      Local User:<asp:Label ID="OptLocalUser" runat="server" /><br/>
     
      RootDSE:
      <asp:GridView ID="GridView1" runat="server">
      </asp:GridView>
     
      AD User Path:<asp:Label ID="OptUserPath" runat="server" /><br/>
      <asp:GridView ID="GridView2" runat="server">
      </asp:GridView>
    </div>
    </form>
</body>
</html>

Is Your Next Programmer a Moron?

We have been interviewing for a .net developer position for quite a while - way too long in fact.  What I thought would be a quick interview process has turned into a very rude awaking.

When I was looking for a job I was contacted by a contractor with a position that interested me.  I was warned however, that the interview process included a 4 hour programming test.  Many bodies had been thrown at this test and very few succeeded.  I try not to be arrogant but I do like to think that I'm good at my job.  Plus I have no certifications and no big name companies on my resume so I usually welcome the chance to prove myself.  The test wasn't rocket science but it wasn't simple either.  It was a simple recursion/stack problem, some database queries, a basic dynamic web page problem, and creating a functional calendar.  It was a lot of time to give a prospective employer but when I got the job I thought the test was well worth it after so many other companies didn't even give me a phone call or an email.

OK, a year later I'm looking to clone myself.  As much as I thought the test was warranted, I also thought that 4 hours is a really long time.  I created a trimmed down version that was nothing more than sql queries and a simple databinding exercise.  After a few candidates we decided it was too data-centric.  My boss forwarded me the fizz-buzz question.  I just couldn't believe what it's author was saying; senior level programmers can't write this trivial block of code?  As more of an experiment than anything, we added in our own variation of fizz-buzz.

And this is where the horror starts.  The very next candidate came in, sat down for 20 minutes, and asked to leave citing "personal reasons" without writing a single line of code.  I couldn't believe it.  I examined all the possibilities: Maybe she didn't understand the mod operator?  I suppose you could do years of CRUD style programming and never need it?  Maybe she couldn't read English?  Maybe she was a VB.net programmer (the test is in C#)?  But then there was always the last option - this 10 page resume couldn't actually write code.

We had another come in and at least attempt it but also failed miserably.  Another came in and passed fizz-buzz, but couldn't write a database query to save his life.  Another came in and did do well on both fizz-buzz and the queries.  He didn't even know ASP.net, but because he at least seemed like he had his head screwed on straight we were willing to train him.  But problems with his contracting house kept him from coming aboard.

I couldn't get over that fizz-buzz relies very heavily on a single operator.  I think it was denial more than anything.  I couldn't help but thinking something is wrong with the test instead of the candidates.   Consequently, I added another simple question.  It only required a boolean be returned if a second phrase could be made from the letters of the first phrase.  I figured if they couldn't do fizz-buzz I'd give them a second attempt.

Originally we had advertised only a 3 month contract and we theorized that the short contract was one of the reasons behind the substandard pool we were pulling from.  We upgraded the position to a 6 month contract to hire and called back a promising applicant that originally turned the short contract down.  I started with him just like I did with everybody else; I explained each question, showed him where the answers were (yes we display what the exact answer should look like), went over any hang-ups that I thought he might have, and reiterated that he can ask any question.  I didn't even get thru explaining my new anagram question when he stopped me and said something like "if this is the what you're looking for I don't think I want to work here."  I reiterated that it's OK if there were parts that he couldn't do.  He didn't care.  I brought him back to my boss and he asked what he didn't like about the test, he referred to it as a bunch of puzzle questions that he needed google for and that we should look at his resume to the systems he's "architected."  I walked him out and he described his (superior) interview process of asking questions about polymorphism and factory patterns.  I just wanted to slap him and say "Don't you understand?  It's people like you why we do this!  Yes you can use some big words and put fancy projects on your resume.  You can call yourself a senior developer and an architect.  But you can't write code!!"

On my way back to my desk I realized my blood was boiling.  I understood why good developers either can't find a job or can't get the pay they deserve.  I understood why so many projects fail.  I understood why I spend so much time cleaning up crap.  I understood why we have been trying to fill a position for 2 months.  It's because it would appear that the ratio of fake to genuine developer resumes out there is about 5:1 or worse!  And the really bad news is that these simple tests don't even guarantee that you have a good programmer.  They only tell you if you have a programmer at all. 

The moral of the story?  If you want to build a house, hire a guy that can hammer a nail.  If you want to build a car, hire a guy that can loosen a bolt.  If you want to build a piece of software, than hire a developer that can write programming 101 code.  Test your programmers! 

Catching 'Thrown' Exceptions in Visual Studio

Every once in a while I'll have an app crash without any warning at all.  One minute it's there, one minute it's not.  No exception message, no break in visual studio while debugging, nothing.  Each time this happens I just allow visual studio to catch 'thrown' exceptions rather than just 'user-unhandled' exceptions.  This means that visual studio will break right at the time it occurs, rather than letting it bubble up to see if it will be handled later in the code.  Go to File>Debug>Exceptions and check the 'Thrown' boxes...

Event-Based Keyboard Wedge Scanner

I just came off a project programming Motorola barcode canners based on Windows Mobile.  The Motorola/Symbol API was top notch and provided access to everything I needed.  This included an event driven API for the scanner.  This meant I had a nice "Scan" event to use from within my code rather than rely on simple keyboard emulation.  This great when a single barcode had multiple fields of data.  I could parse everything behind the scenes without using a "capture" field on my forms.

The next project also needed a barcode scanner but would be a Winforms app rather than a mobile app.  To my surprise the APIs for these usb and serial scanners were not nearly as good.  Only a select few scanners supported Motorola's usb SNAPI interface, and there was no native .net support there (just a bunch of DllImports).  Others had an RS232 based interface but it wasn't much better than reading raw data from the serial port.  A few of the scanners supported the OPOS standard which means I could have used POS for .net but that seemed overkill for my needs.

Finally I realized there was a much simpler solution.  Most of these scanners supported a keyboard emulation mode.  More importantly they supported the ability to be programmed to include prefixes and suffixes in the scan.  While most would use this feature to send an 'Enter' after a successful scan, I realized I could use this to send special characters to indicate the start and end of a scan and separate it from normal keyboard input.  From there making my own API was simple.

There were two main points to this.  The first was to monitor keypresses:

HostForm.KeyPreview = true;
this.enabled = true;
if (this.enabled)
{
  HostForm.KeyPress += new KeyPressEventHandler(KeyPressHandler);
}
else
{
  HostForm.KeyPress -= KeyPressHandler;
}

The line of interest here should be the Form.KeyPreview=true.  This allows you to pickup keyboard strokes at the form level rather than from the control with focus.

From there it's just a matter of looking for the started and ending characters (i chose ctrl-s and ctrl-t, ascii char 19 and 20 resp):

public void KeyPressHandler(object sender, KeyPressEventArgs e)
{
  //if a scan has started...
  if (!IsScanning && e.KeyChar == ScanStartCharacter)
  {
    IsScanning = true;
    timeoutTimer.Start();
    e.Handled = true;
    OnScanStarted();
  }
  //if a scan is in progress and is not being terminated...
  else if (IsScanning && e.KeyChar != ScanEndCharacter)
  {
    timeoutTimer.Stop();
    //...then add the char to the buffer
    scanBuffer.Append(e.KeyChar);

    timeoutTimer.Start();
    e.Handled = true;
  }
  //if a scan has ended
  else if (e.KeyChar == ScanEndCharacter)
  {
    timeoutTimer.Stop();

    CompleteScan(false);

    e.Handled = true;
  }
  else  //not scanning, pass the character on as keyboard input.
  {
    e.Handled = false;
  }
}

I threw a timer in there just to make sure the scan eventually completes, even if the user forgets to program the suffix into the scanner.

That's pretty much it.  The full code is attached.

Easy UI Thread Management with System.Timers.Timer

There's a great article on msdn that covers the 3 different types of timers.  I've basically boiled it down to this:

  • Use the Forms.Timer for simple updates to the UI.  For example i like to bold certain errors when they happen and then return them to a normal font.  This helps the user identify when they caused the same error twice.
  • Use the Threading.Timer to execute one-off processes off of the UI thread
  • Use the Timers.Timer to execute recurring processes.  This timer's AutoReset and Enabled properties make it much cleaner to use than Threading.Timer's Change method.

I tended to lean to Threading.Timer in most cases.  When I needed to do something on the UI thread i just used Invoke.  What I missed from the article is Timers.Timer's SynchronizingObject property.  Simply set this to your form or something else on your UI thread, and no "InvokeRequired" code is needed to access the UI.  Because of this, this timer has no become my timer of choice.  The only drawback to it compared to the Threading version, is that the Threading version's callback takes an object parameter allowing you to pass some state into the callback.  That's only a minor issue considering you can either create a field to temporarily store this additional data or you can create different "Elapsed" events.

Some other basic threading tips:

  • Don't forget to hold a reference somewhere to your timer.  If there are no references, the timer might be garbage collected before it gets a chance to fire
  • Timers.Timer AutoReset is true by default.  I almost always set this to false and then reset Enabled=true in the Elapsed event.  This avoids any reenterance issues.

Missing 'out' parameter on a web service proxy class

After adding a "web service reference" in visual studio and instantiating the generated proxy class, I noticed I was a parameter short.  The web service method had three parameters - a custom class, an 'out' int and an 'out' string.  But the proxy class only had the first and the last of those three parameters.  I took a quick look at the WSDL and that was correct.  I had also added a reference to a similar service the week before so I knew that you could have multiple 'out' parameters.  After a little googling I found this article.  Because my web method returned 'void' the proxy class generator was using the first 'out' param as the return value instead.  I just returned a Boolean instead and that made my first out parameter show up as a parameter in the proxy class.

Executing SQL Plus Batches with Bind Variables from .Net

In an earlier post I talked about bind variables in SQL Plus and PL/SQL and only linked to an OTN forum post to show how these bind variables related to .net.  I didn't realize there were a couple of mistakes in that post so i thought I go thru the entire process here.

Using bind variables normally doesn't amount to much more than using the SQL parameters you're probably already familiar with.  The only difference I can think of is the use of a colon instead of the '@' character and that you only use the colon when using it in a parameterized query (you do not use it for stored proc params).  However, with very little SQL Plus experience, i found executing a batch of statements from .net a little tricky.

I suggest starting with a raw SQL Plus / PL/SQL batch and get that working.  In the following example, i wanted to call a stored proc with fewer params than it asked for which required a lookup in a separate table.

var badge_no varchar2(15);
var p_return_success number;
var p_return_string varchar2(4000);
exec :badge_no := '1234';
exec :p_return_success := -1
declare
  l_myKey varchar(50);
begin
  select rf_key into l_myKey from rf_security where badge_no=:badge_no;
  CHECK_BADGE(
    P_BADGE_NO => :badge_no,
    P_KEY => l_myKey,
    P_RETURN_SUCCESS => :p_return_success,
    P_RETURN_STRING => :p_return_string);
exception
  when NO_DATA_FOUND then
    :p_return_success := 0;
    :p_return_string := 'User not found';
end;
/
print badge_no
print p_return_success
print p_return_string

For those of you that aren't familiar with SQL Plus and PL/SQL, the PL/SQL portion of this starts with the 'declare' and ends with end; 

Note: One minor thing that bit me.  If you're using SQL Developer as your query tool, I found that it's a bit buggy with bind variables and output.  For example, I originally had the ':p_return_success := -1 ' inside of the 'begin' block.  That worked as expected in SQL Plus but not in SQL Developer (which is much easier to use).

When you add a parameter like this:

cmd.Parameters.Add("P_BADGE_NO", OracleDbType.Varchar2).Value = AuthHeader.Token.UserId.ToString();

That gets translated into 'var badge_no varchar2' by the provider.  So the statement trimmed down for .net is (assuming you're using Oracle.OracleClient from ODP.net and not System.Data.OracleClient):

      OracleCommand cmd = new OracleCommand(
        @"
        declare
          l_myKey varchar(50);
        begin
          select rf_key into l_myKey from rf_security where badge_no=:badge_no;
          CHECK_BADGE(
            P_BADGE_NO => :badge_no,
            P_KEY => l_myKey,
            P_RETURN_SUCCESS => :p_return_success,
            P_RETURN_STRING => :p_return_string);
        exception
          when NO_DATA_FOUND then
            :p_return_success := 0;
            :p_return_string := 'User not found,';
        end;
        ";
      cmd.Parameters.Add("P_BADGE_NO", OracleDbType.Varchar2).Value = AuthHeader.Token.UserId.ToString();
      cmd.Parameters.Add("P_RETURN_SUCCESS", OracleDbType.Decimal).Direction = ParameterDirection.Output;
      cmd.Parameters.Add("P_RETURN_STRING", OracleDbType.Varchar2, 4000).Direction = ParameterDirection.Output;

      using (cmd.Connection = new OracleConnection(AppConfig.General.ConnectionString))
      {
        cmd.Connection.Open();
        cmd.ExecuteNonQuery();
      }

Notice that we left l_myKey variable in.  This variable is only used locally within the batch and we do not need it's value to go in or out.  That's why we declared it as pl/sql variable inside of the declare instead of a SQL Plus variable outside.

However, as is, this will blow up with the following error:

ORA-06550: line 1, column 1:
PLS-00103: Encountered the symbol "" when expecting one of the following

It turns out that the very old SQL Plus has a problem with the windows newline.  I just added a .Replace to my string and it worked from there.

      OracleCommand cmd = new OracleCommand(
        @"
        declare
          l_myKey varchar(50);
        begin
          select rf_key into l_myKey from rf_security where badge_no=:badge_no;
          CHECK_BADGE(
            P_BADGE_NO => :badge_no,
            P_KEY => l_myKey,
            P_RETURN_SUCCESS => :p_return_success,
            P_RETURN_STRING => :p_return_string);
        exception
          when NO_DATA_FOUND then
            :p_return_success := 0;
            :p_return_string := 'User not found,';
        end;
        ".Replace(Environment.NewLine, " "));

 Good Luck,

B