Brothers In Code

...a serious misallocation of .net resources

Cool Way to Generate a Date Dimension Table in Oracle

Say you want a weekly sales report grouped by each day and you want to show days with zero sales. Usually this is accomplished by creating and joining to a date table that has one row for each day. If you're going to be doing this report all the time or need additional groupings like quarter or fiscal year, than it makes sense to create this table. But sometimes I need to do this for one report in a database that doesn't already have this table. For oracle, Uli Bethke describes what I think is the best way to date. In short, it's clever use of oracle's 'connect by' clause to recursively join dual to itself. For example, the following would generate all days in march:

SELECT
trunc(to_date('1-mar-2009','dd-mon-yyyy')) + NUMTODSINTERVAL(n,'day') AS Full_Date
FROM (
select (level-1) n
from dual
connect by level-1 <= trunc(to_date('31-mar-2009','dd-mon-yyyy')) - trunc(to_date('1-mar-2009','dd-mon-yyyy'))
);

Is Your Next Programmer a Moron?

We have been interviewing for a .net developer position for quite a while - way too long in fact.  What I thought would be a quick interview process has turned into a very rude awaking.

When I was looking for a job I was contacted by a contractor with a position that interested me.  I was warned however, that the interview process included a 4 hour programming test.  Many bodies had been thrown at this test and very few succeeded.  I try not to be arrogant but I do like to think that I'm good at my job.  Plus I have no certifications and no big name companies on my resume so I usually welcome the chance to prove myself.  The test wasn't rocket science but it wasn't simple either.  It was a simple recursion/stack problem, some database queries, a basic dynamic web page problem, and creating a functional calendar.  It was a lot of time to give a prospective employer but when I got the job I thought the test was well worth it after so many other companies didn't even give me a phone call or an email.

OK, a year later I'm looking to clone myself.  As much as I thought the test was warranted, I also thought that 4 hours is a really long time.  I created a trimmed down version that was nothing more than sql queries and a simple databinding exercise.  After a few candidates we decided it was too data-centric.  My boss forwarded me the fizz-buzz question.  I just couldn't believe what it's author was saying; senior level programmers can't write this trivial block of code?  As more of an experiment than anything, we added in our own variation of fizz-buzz.

And this is where the horror starts.  The very next candidate came in, sat down for 20 minutes, and asked to leave citing "personal reasons" without writing a single line of code.  I couldn't believe it.  I examined all the possibilities: Maybe she didn't understand the mod operator?  I suppose you could do years of CRUD style programming and never need it?  Maybe she couldn't read English?  Maybe she was a VB.net programmer (the test is in C#)?  But then there was always the last option - this 10 page resume couldn't actually write code.

We had another come in and at least attempt it but also failed miserably.  Another came in and passed fizz-buzz, but couldn't write a database query to save his life.  Another came in and did do well on both fizz-buzz and the queries.  He didn't even know ASP.net, but because he at least seemed like he had his head screwed on straight we were willing to train him.  But problems with his contracting house kept him from coming aboard.

I couldn't get over that fizz-buzz relies very heavily on a single operator.  I think it was denial more than anything.  I couldn't help but thinking something is wrong with the test instead of the candidates.   Consequently, I added another simple question.  It only required a boolean be returned if a second phrase could be made from the letters of the first phrase.  I figured if they couldn't do fizz-buzz I'd give them a second attempt.

Originally we had advertised only a 3 month contract and we theorized that the short contract was one of the reasons behind the substandard pool we were pulling from.  We upgraded the position to a 6 month contract to hire and called back a promising applicant that originally turned the short contract down.  I started with him just like I did with everybody else; I explained each question, showed him where the answers were (yes we display what the exact answer should look like), went over any hang-ups that I thought he might have, and reiterated that he can ask any question.  I didn't even get thru explaining my new anagram question when he stopped me and said something like "if this is the what you're looking for I don't think I want to work here."  I reiterated that it's OK if there were parts that he couldn't do.  He didn't care.  I brought him back to my boss and he asked what he didn't like about the test, he referred to it as a bunch of puzzle questions that he needed google for and that we should look at his resume to the systems he's "architected."  I walked him out and he described his (superior) interview process of asking questions about polymorphism and factory patterns.  I just wanted to slap him and say "Don't you understand?  It's people like you why we do this!  Yes you can use some big words and put fancy projects on your resume.  You can call yourself a senior developer and an architect.  But you can't write code!!"

On my way back to my desk I realized my blood was boiling.  I understood why good developers either can't find a job or can't get the pay they deserve.  I understood why so many projects fail.  I understood why I spend so much time cleaning up crap.  I understood why we have been trying to fill a position for 2 months.  It's because it would appear that the ratio of fake to genuine developer resumes out there is about 5:1 or worse!  And the really bad news is that these simple tests don't even guarantee that you have a good programmer.  They only tell you if you have a programmer at all. 

The moral of the story?  If you want to build a house, hire a guy that can hammer a nail.  If you want to build a car, hire a guy that can loosen a bolt.  If you want to build a piece of software, than hire a developer that can write programming 101 code.  Test your programmers! 

From ASMX to WCF 4.0 - Second Attempt

Introduction

I admit, I'm very slow when it comes to absorbing new technologies, frameworks, or methodologies.  I have a very simple 3-teir pattern that works very well and I've seen way too many debocles with projects that jumped head first into the latest craze and then everybody shrugs their shoulders when asked what went wrong.  SOA and webservices, ORMs, Workflow Foundation,  I've seen them all find their way into a project that would have faired much better with a more traditional platform.  If you want to know if you're next project is going to be another IT failure statistic, then I would look carefully at the technologies you're including for the sake of being on the cutting edge.

And so now you have my reason for my delayed start into WCF.  This is actually my second attempt.  I had starting using it as the middle teir for a Windows Mobile project, but quickly found it's complexity overkill for something that didnt' support most of it's features anyway.  I went back to good ole ASMX services with a custom authentication header.  But now another service project has come up.  It has the additional requirement of being publicly accessible, so I've decided to give WCF another chance for the sake of newer standards and better interoperability.  And this time I really gave it the time that was needed.  Which translates to: it wasn't easy.

The flexibility of WCF is impressive, which is another way of saying the configurability is daunting.  I read an article that compared ASMX to a small Cessna and WCF to a 747.  I will warn you now - if you're project only requires a 'flight' from Detroit to Chicago, then avoid the debocles I mentioned above and take the Cessna (ASMX).  Most of the things I struggled configuring for the first time in WCF, I could have programmed faster with an ASMX service.  But at some point I suppose my knowledge of this technology will catch up to it's underlying concepts.

Ok so enough babble.  Rather than write a neat and tidy article that makes me out to be a genious that does no wrong and make WCF look like a piece of cake, I'm going to write this article as it happened so I might help people out of the same holes that I fell in.

Hello World

I started out with a simple service that took two parameters and returned an object.  There's a dozen tutorials that will walk you through the ServiceContract and DataContract attribs to do this, but I just renamed the default items created with a new WCF project inside of Visual studio 2010 and added in my own data.  When following such a tutorial I ran into my first problem:

The 'Edit WCF Configuration' menu optionis not available when right clicking on the web.config. 

I could go to Tools > WCF Service Configuration Editor, but doing so brought up an empty config.  However, once I opened that, the 'Edit WCF Configuration' option was now available on my web.config.

That was simple enough.  I pressed F5, and the WCF Test Client started up and I tested my service.  I then added another file to the project.  Next problem:

Debugging with F5 started a browser with the service page rather than the WCF test client. 

Marking the page I wanted to test as the 'Start Page' fixed that.  However, it wasn't long before I got sick of waiting for the test client to start, so eventually I started the executable for the test client manually and just used it's 'Refresh' option after each compile.

Transport Level Custom Authentication

But that was the end of simple.  I knew my services were going to require custom authentication so that was the next task.  This article makes this look like a simple task.  I followed it but that's where my real education with WCF began.  I first configured this service for transport security and that gave me my next error:

"Could not find a base address that matches scheme https for the endpoint with binding basichttpbinding."  \

Configuring <security mode="Transport"> automatically configures your endpoint address to be https.   Ok, so I tried to disable that by setting <security mode="TransportCredentialOnly"> and that brought me to:

"Security settings for this service require 'Basic' Authentication but it is not enabled for the IIS application that hosts this service."

Well the ASP.net dev server doesn't have a basic auth mode so at this point I switched to my local IIS server and enabled basic auth for the site.

"BasicHttp binding requires that BasicHttpBinding.Security.Message.ClientCredentialType be equivalent to the BasicHttpMessageCredentialType.Certificate credential type for secure messages. Select Transport or TransportWithMessageCredential security for UserName credentials."

Apparently if you set the transport layer to send credentials only and forget the ssl, WCF says 'Whoa, you can't send plaintext user names and passwords' and it wants to do the certificate encryption itself.  At first this made no sense to me.  Why does WCF care?  SSL has always been handled by IIS OUTSIDE of the application.  But then I realized that WCF can be self-hosted or hosted in windows service.  In fact it probably should be since the only features that IIS adds is app pool recycling and auto-startup and I'm not sure either of those are worth the problems I ran in to.  However, i wanted to avoid going outside the status quo for webservices (it had more to do with my stubborness), so I continued down the IIS route.

There is a new option called allowInsecureTransport (also availabile as a hotfix in 3.5 sp1) that is supposed to allow this.  Not only is this desirable in terms of development, but many organizations terminate the ssl connection at the firewall or an ssl accelerator.  It also erks the heck out of me that it took nearly 4 versions to realize this.  It erks me even more that it breaks the WSDL generation and requires a hack to fix.  Ok fine, I generated a self-signed cert and installed it on my IIS server.

The funny thing is that after all this I realized there is a note in custom authentication tutorial that says you can't configure a service for transport security if you're going to host it in IIS.  This makes perfect sense.  HTTP-Basic authentication is handled and validated against the server or active directory, long before the request is passed to IIS.  Uggg!

Transport Level Message Level Custom Authentication

So at this point I decided to abandon passing the credentials via http and instead include them in the message.  This meant setting the following in my basicHTTPBinding:


 <security mode="TransportWithMessageCredential">
  <message clientCredentialType="UserName"/>
</security>

From there I could at least bring up the web service info page in a browser without error.  However, the WCF test client cannot pass credentials, so I whipped up a quick nunit project instead.  I added a service reference to my WCF service and added this quick test method:


[Test]
public void GetInventoryDetail()
{
  ItemServiceClient wsClient = new ItemServiceClient();
  wsClient.ClientCredentials.UserName.UserName = "test1";
  wsClient.ClientCredentials.UserName.Password = "1tset";
  ItemInventoryDetail data = wsClient.GetInventoryDetail("64817", "11006.00-G");
  Assert.AreEqual("11006.00-G", data.ItemNumber);
}

I then got:

System.ServiceModel.Security.SecurityNegotiationException : Could not establish trust relationship for the SSL/TLS secure channel with authority 'mycomputername'.

This was because, my computer as a certificate authority is  not trusted.  So I added the following to the top of my test method...


  System.Net.ServicePointManager.ServerCertificateValidationCallback += (se, cert, chain, sslerror) =>
  {
    return true;
  };

I then got:

System.ServiceModel.Security.MessageSecurityException : An unsecured or incorrectly secured fault was received from the other party. See the inner FaultException for the fault code and detail.
  ----> System.ServiceModel.FaultException : At least one security token in the message could not be validated.

This didn't mean a whole lot so I enabled tracing for the service by adding the following to the web.config:


  <system.diagnostics>
    <sources>
      <source name="System.ServiceModel"
              switchValue="All"
              propagateActivity="true">
        <listeners>
          <add name="traceListener"
              type="System.Diagnostics.XmlWriterTraceListener"
              initializeData= "c:\temp\WCFTraces.svclog" />
        </listeners>
      </source>
    </sources>
  </system.diagnostics>

Opening the tracefile, I found the following exceptions:

System.ServiceModel.Security.MessageSecurityException: Message security verification failed. --->; System.IdentityModel.Tokens.SecurityTokenValidationException: LogonUser failed for the 'test1' user. Ensure that the user has a valid Windows account. --->; System.ComponentModel.Win32Exception: Logon failure: unknown user name or bad password

Win32Exception?  A helpful msdn poster then told me that I didn't link my behavior to my service:

<service behaviorConfiguration="CustomAuthenticationBehavior"

This part was left out of the tutorial I was following as well.  Another solution, was to remove the behavior name from the behavior so that the two could be matched on an empty string.

And after all that, it finally worked!

So after two attempts I'm finally sitting here with a basic working model for a WCF project.  I still need to do a little research as far as structuring a service (many classes end up becoming a single service), but I have a lot more confidence this time to actually move forward with WCF.  Am I going to use it for everything?  No.  But I have to admit I enjoy the idea of writting a local business service and later exposing it to the web with just a simple configuration change.

B

Events in Global.asax Do Not Fire - Did You Forget to Drop the File?

I was working in a web service project where by default, the codebehind is compiled into a single dll.  I didn't expect to need the Global.asax file that went with it's corresponding .cs file, but it turns out that the events won't fire without it.

Unwanted Assembly/EXE Sharing in Windows Mobile

I just got done merging two Windows Mobile applications into one and a user complained that the combined version was missing a feature of one of the original apps.  As I set out to prove him wrong I bumped into something very wierd.  I set a breakpoint in the event in the old app that I was expecting to check and nothing happened.  The form in the app did absolutely nothing.  However, I was pushing all http traffic from the mobile device thru fiddler and I could see that the form was hitting the webservice that the new combined app was supposed to be using.  The device also gave the beep code I had programmed in after a successful operation.  I put a breakpoint in a couple of other methods like Form.Load and Form.Close and even a couple of Button.Click events and those I was able to debug.  I wasn't loading any assemblies in the GAC but even if I was I couldn't explain this partial debugging ability.  The new app was on the device but it was in a separate folder so at first I disregarding the possibility that I was getting some sort of cross contamination.  But then I thought what if the other app is actually running?  Sure enough, after checking task manager, I could see the new app was running in the background and killing it let me debug the old app.  How did this happen?  The classes do have the same namespace and name but I've never heard of an app sharing another app's code in memory.

Wrong Tab Order in Windows Mobile

I had a form that didn't seem to want to follow the TabIndex that I had set on a number of controls.  The cursor jumped around passed controls with a lower index and then would come back to them later.  The difference between this form and others was that this form had a panel on it that hid a couple of controls until they were displayed upon the click of a radio button.  I did notice that if I deleted the controls and recreated them in the order I wanted them to tab, they would tab correctly.  However, starting over is a lousy fix so I dug into it a bit deeper.  After debugging I found that the TabIndex on the controls within the panel were being reset.  It also turns out that even though a Panel control has no mention of a TabIndex property in the Properties pane or with intellisense, it does in fact have that property.  Setting the Panel's TabIndex in the form load event fixed the issue for me.  To make it so I wouldn't get bit by this again,  I set the panel's TabIndex to the index of the first control in the panel.

B

Catching 'Thrown' Exceptions in Visual Studio

Every once in a while I'll have an app crash without any warning at all.  One minute it's there, one minute it's not.  No exception message, no break in visual studio while debugging, nothing.  Each time this happens I just allow visual studio to catch 'thrown' exceptions rather than just 'user-unhandled' exceptions.  This means that visual studio will break right at the time it occurs, rather than letting it bubble up to see if it will be handled later in the code.  Go to File>Debug>Exceptions and check the 'Thrown' boxes...

Event-Based Keyboard Wedge Scanner

I just came off a project programming Motorola barcode canners based on Windows Mobile.  The Motorola/Symbol API was top notch and provided access to everything I needed.  This included an event driven API for the scanner.  This meant I had a nice "Scan" event to use from within my code rather than rely on simple keyboard emulation.  This great when a single barcode had multiple fields of data.  I could parse everything behind the scenes without using a "capture" field on my forms.

The next project also needed a barcode scanner but would be a Winforms app rather than a mobile app.  To my surprise the APIs for these usb and serial scanners were not nearly as good.  Only a select few scanners supported Motorola's usb SNAPI interface, and there was no native .net support there (just a bunch of DllImports).  Others had an RS232 based interface but it wasn't much better than reading raw data from the serial port.  A few of the scanners supported the OPOS standard which means I could have used POS for .net but that seemed overkill for my needs.

Finally I realized there was a much simpler solution.  Most of these scanners supported a keyboard emulation mode.  More importantly they supported the ability to be programmed to include prefixes and suffixes in the scan.  While most would use this feature to send an 'Enter' after a successful scan, I realized I could use this to send special characters to indicate the start and end of a scan and separate it from normal keyboard input.  From there making my own API was simple.

There were two main points to this.  The first was to monitor keypresses:

HostForm.KeyPreview = true;
this.enabled = true;
if (this.enabled)
{
  HostForm.KeyPress += new KeyPressEventHandler(KeyPressHandler);
}
else
{
  HostForm.KeyPress -= KeyPressHandler;
}

The line of interest here should be the Form.KeyPreview=true.  This allows you to pickup keyboard strokes at the form level rather than from the control with focus.

From there it's just a matter of looking for the started and ending characters (i chose ctrl-s and ctrl-t, ascii char 19 and 20 resp):

public void KeyPressHandler(object sender, KeyPressEventArgs e)
{
  //if a scan has started...
  if (!IsScanning && e.KeyChar == ScanStartCharacter)
  {
    IsScanning = true;
    timeoutTimer.Start();
    e.Handled = true;
    OnScanStarted();
  }
  //if a scan is in progress and is not being terminated...
  else if (IsScanning && e.KeyChar != ScanEndCharacter)
  {
    timeoutTimer.Stop();
    //...then add the char to the buffer
    scanBuffer.Append(e.KeyChar);

    timeoutTimer.Start();
    e.Handled = true;
  }
  //if a scan has ended
  else if (e.KeyChar == ScanEndCharacter)
  {
    timeoutTimer.Stop();

    CompleteScan(false);

    e.Handled = true;
  }
  else  //not scanning, pass the character on as keyboard input.
  {
    e.Handled = false;
  }
}

I threw a timer in there just to make sure the scan eventually completes, even if the user forgets to program the suffix into the scanner.

That's pretty much it.  The full code is attached.

Conditionally Opening a Modal Form in a Parent Form's Activate Event

In a Winforms App, I wanted to see if a local configuration file existed, and if it didn't I wanted to force the user into the setup form after being prompted to do so.  I basically had the following code in my main form:

private void MainForm_Activated(object sender, EventArgs e)
{
  //if no config has been loaded...
  if (SaveData.Current.ScannerConfigId == 0)
  {
    DialogResult result = MessageBox.Show(
      "No local configuration was found.  Press OK to select a configuration.  Press Cancel to exit.",
      "Setup",
      MessageBoxButtons.OKCancel);
    if (result == DialogResult.OK)
    {
      //...then force the user into the setup form
      Form form = new SetupForm();
      form.Owner = this;
      form.ShowDialog();
    }
    else
      Application.Exit();

  }
}

Unfortunately, I found that upon clicking 'OK' to close the MessageBox, the activate event was fired again before completing the first one that opened the MessageBox.  To fix it I just added a simple semaphore:

private void MainForm_Activated(object sender, EventArgs e)
{
  if (!checkingConfig)
  {
    //if no config has been loaded...
    if (SaveData.Current.ScannerConfigId == 0)
    {
      checkingConfig = true;
      DialogResult result = MessageBox.Show(
        "No local configuration was found.  Press OK to select a configuration.  Press Cancel to exit.",
        "Setup",
        MessageBoxButtons.OKCancel);
      checkingConfig = false;
      if (result == DialogResult.OK)
      {
        //...then force the user into the setup form
        Form form = new SetupForm();
        form.Owner = this;
        form.ShowDialog();
      }
      else
        Application.Exit();

    }
  }
}

After that, the second call quickly exited allowing the first to pick up where it left off.

Easy UI Thread Management with System.Timers.Timer

There's a great article on msdn that covers the 3 different types of timers.  I've basically boiled it down to this:

  • Use the Forms.Timer for simple updates to the UI.  For example i like to bold certain errors when they happen and then return them to a normal font.  This helps the user identify when they caused the same error twice.
  • Use the Threading.Timer to execute one-off processes off of the UI thread
  • Use the Timers.Timer to execute recurring processes.  This timer's AutoReset and Enabled properties make it much cleaner to use than Threading.Timer's Change method.

I tended to lean to Threading.Timer in most cases.  When I needed to do something on the UI thread i just used Invoke.  What I missed from the article is Timers.Timer's SynchronizingObject property.  Simply set this to your form or something else on your UI thread, and no "InvokeRequired" code is needed to access the UI.  Because of this, this timer has no become my timer of choice.  The only drawback to it compared to the Threading version, is that the Threading version's callback takes an object parameter allowing you to pass some state into the callback.  That's only a minor issue considering you can either create a field to temporarily store this additional data or you can create different "Elapsed" events.

Some other basic threading tips:

  • Don't forget to hold a reference somewhere to your timer.  If there are no references, the timer might be garbage collected before it gets a chance to fire
  • Timers.Timer AutoReset is true by default.  I almost always set this to false and then reset Enabled=true in the Elapsed event.  This avoids any reenterance issues.