As a migration company we help many, who are used to developing in traditional scripting or batch JCL languages, as they make the leap to full OO scripting language on distributed systems. While all JCL developers succeed in the adjustment with time, the first step always feels like diving in at the deep end. In this blog post I’ll be going over some of the pitfalls and quirks that I found in PowerShell while learning the language and creating our scripting and batch migration solutions.

Intro

The aim of writing this is to save people who are making the transition to PowerShell from some of the head-scratching moments that I’ve had, because I’m nice like that (also because they told me I had to write something this month).

Some context

The past two years we’ve seen more companies showing interest in full .NET stack solutions, for example:

  • For migrating applications with JCL and NATURAL to PowerShell and C#
  • For migrating applications with KornShell and COBOL to PowerShell and NET COBOL

The common part in both being the fact that PowerShell was chosen as the target scripting language. This makes a lot of sense: it’s what Microsoft is investing in as its scripting solution and its integration with other Microsoft products and admin tools (think WMI) will only grow. It’s a .NET language, and so can use CLS compliant libraries from any other .NET language. It’s a lot more useful than the old Windows batch command files, and a lot more native than using Cygwin.

One of the good things about Anubex migration solutions (if I do say so myself) is that they make the transition from the original language to the target language as smooth as possible by making it as one-one as possible. For example in our JCL solution each JCL command becomes a PowerShell cmdlet with the same basic structure of command name, and same-named or positioned operands. For example:

// DLBL          CMWKF02,'VK009-B2.FIRMA01',,VSAM,CAT=UCAT

Becomes

Set-VseDiskLabel CMWKF02 'VK009-B2.FIRMA01' $null VSAM -Cat UCAT

 

Now it’s perfectly possible to continue writing JCL-like scripts and never have to delve into the complexities of PowerShell or the .NET framework, however I’m sure that once given such a large tool kit developers will want to start extending or re-writing their scripts with native PowerShell and .NET code. Since in using this toolkit you will inevitably drop the hammer on your toes, I’ve written the following to:

  1. Show you you’re not alone ;-)
  2. Help you avoid one or two of these mishaps!

Enjoy.

Issue 1: It’s all unravelling

The first feature I’d like to discuss is that of array flattening or unravelling. This is a sometimes nice, sometimes deceptive feature of PowerShell, in which it will flatten arrays (including nested arrays) so that the objects can be processed nicely within a pipeline. Also arrays with one item are ‘tidied up’ so we just get the object contained within the array.

Unfortunately this isn’t always desirable.

function foo 
{ 
  return @("abc", "def", "ghi") 
}
$x = foo
$x[0]

Gives us, as expected the first string in the array

abc

 However a single item in the array.

function foo 
{ 
  return @("abc") 
}
$x = foo
$x[0]

Gives us…

a

And suddenly we are indexing into the first string without meaning to.

I’ve had cases where I was reading a file and taking the first line. This worked fine for several weeks until on a particular day the file only contained that one line, and then suddenly it broke.

Fortunately there is a way to say to PowerShell, no I really did want that array back thank you very much. Simply adding a leading comma in front of the array will avoid this behavior and ensure you always get what you expect. 

function foo{ 
  return ,@("abc") 
}
$x = foo
$x[0]

Gives us, as desired, the first item in the array

abc

Issue 2: Fun with functions – Pass it on

PowerShell is all about object and pipelines. Functions are also made to be used in pipelines. But you have to be careful with them. Let’s use our trusty foo function again.

function foo 
{ 
  return @("abc", "def", "ghi") 
}
$x = @()
foo | % { $x += $_ } 

This calls foo in a pipeline. The output values are all passed down the pipeline, and used by the Foreach-Object cmdlet which just adds them to another array. If we look at $x we see it now contains:

abc
def
ghi 

Now let’s say we add some debugging to foo.

function foo 
{ 
  echo “I’m in foo”
  return @("abc", "def", "ghi") 
}
$x = @()
foo | % { $x += $_ } 
Running the command again

Running the command again, we don’t see our echo. Humm strange? Let’s look inside $x

I’m in foo
abc
def
ghi 

Yep, it contains our message. Echo is an alias for the Write-Output cmdlet. Since we are in a pipeline here, the ‘output’ goes to the next command in the pipeline. This can be confusing to people who expect that only a return should be returning something. In fact the return keyword is completely unnecessary. The function takes anything output and sticks it into an object array which it returns (unless it gets flattened of course!). When in a pipeline each item in the array is passed down the pipeline.

If you want to write something to go to the console while in a pipeline, there are a couple of different ways.

There is the Write-Host cmdlet, assuming the ‘Host’ is a console.

Write-Host "FOO"

FOO 

Or if you’re passing on output from somewhere else, the Out-Host echo.

echo “I’m in foo” | Out-Host 

I’m in foo 

Or you could use a .NET class directly.

[System.Console]::WriteLine("FOO")

FOO 

So be careful what you do, and what you call in a function, as that output will always be passed on.

Issue 3: Fun with functions II – The return of functions

The next things I’d like to discuss are a nice PowerShell function gotcha, and the return value handling. Consider the following:

function a
{
  return 2
}


function b
{
  return 2
} 

Looks pretty straightforward right? We’ve defined two functions ‘a’ and ‘b’, each of which we want to return a number. Now let’s do something with these functions:

if (a -eq b) {
  echo "EQUAL!"
}
else {
  echo "NOT EQUAL!"
} 

Running this gives:

EQUAL!

2 equals 2 right? (If this surprises you then I suggest you stop reading here) So it appears as if the code above is behaving as we would expect. Let’s change one of the functions and see what happens…

function b
{
  return 1
} 

Now running the same if test gives…

EQUAL!

So, do we have the classic mathematical fallacy that 1 = 2? I’m afraid not, something else is happening here.

This is actually calling the function a, and passing it the parameters –eq and b! The array is returned and since the array contains 2 (and is hence non-zero) it evaluates as true!

To be valid this should be:

if ((a) -eq (b)) {
  echo "EQUAL!"
}
else {
  echo "NOT EQUAL!"
} 

So here’s where I want to talk about array flattening again here. Remember we said that functions always return an array of objects? If there was no array flattening helping us out here, we would be comparing one array containing 2 with another array containing 2, which has very different behavior to comparing two integers (it would actually be checking if the LHS array contains the RHS array object, and if so returning it).

While it’s true that the return value of PowerShell functions and cmdlets is always an object array, remember that we are in .NET now. You can write methods in any language, such as C# and call these from PowerShell and they will have the return value type as defined in the C# method. In addition you can even write PowerShell cmdlets in C# if you so wish!

Issue 4: Errors with handling errors

Not really an issue per-se but for new users error handling in PowerShell can be a little confusing and hence error prone. You have the .NET OO style of error handling, with try-catches and exceptions, but also standard error stream (and now warning, verbose, debug,… streams), and (for the old school shell people?) trap blocks. In addition native and external command return codes are handled differently.

So what should you use when? Well first we should make the distinction between native and external commands. Basically native is anything running within the same process while as external processes are things like executables you would start which run in a separate process.

External commands

External commands will write their output to standard output, their errors to the standard error stream and their return code is available in the $LASTEXITCODE variable.

ftp.exe -s:fake.file 2>&1 | Out-Null
$LASTEXITCODE

2 

In the example above we run the FTP executable with a file that doesn’t exist. We use the 2>&1 operator to send standard error to standard output and we pipe the lot to the Out-Null cmdlet to discard it (same as /dev/null or NUL). We display the $LASTEXITCODE which is now set to 2 after the ftp has run.

Native commands

Native commands such as .NET library methods will throw exceptions for critical errors. While cmdlets also write to standard error and warning streams. The success or failure of running a cmdlet can be checked by seeing if the value of $? is true (success) or false (failure) (this only applies to the last cmdlet if you are in a pipeline). In addition the behavior of most cmdlets can be changed by using the –ErrorAction flag, for example to ignore errors on certain cmdlets.

Example 1: Using a .NET method to delete a non-existing directory try

try {
  [System.IO.Directory]::Delete("foo") 
} 
catch [System.IO.IOException] {
  "Something went wrong deleting foo"
}

Something went wrong deleting foo

Example 2: A cmdlet writing to the error stream

(Get-Content foo* | Measure-Object).Count
Get-Content : An object at the specified path foo* does not exist, or has been filtered by the 
              -Include or -Exclude parameter.
At line:1 char:2
+ (Get-Content foo* | Measure-Object).Count
+  ~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (System.String[]:String[]) [Get-Content], 
                              Exception
    + FullyQualifiedErrorId : ItemNotFound,Microsoft.PowerShell.Commands.GetContentCommand
 
0 

Example 3: As example 2, but ignoring errors using the ErrorAction flag

(Get-Content -ErrorAction SilentlyContinue foo* | Measure-Object).Count
 
0 

PowerShell also has a handy array variable called $error which contains all errors which have occurred, even those which have been ignored.

General guidelines

So the general rules would be:

  • Don’t use trap, use the more native try-catch blocks
  • Make sure anything that can throw has a try-catch and the error is handled correctly
  • Always check the return code of externals
  • If something writes to the error stream make sure this is logged correctly
  • If a cmdlet error is not relevant feel free to ignore it!

Conclusion

I hope you’ve enjoyed reading this post, and that it might just help you with some of the troubles you can have when getting into the language. While we’ve discuss problems writing PowerShell code here I can assure you it’s not all bad, and when you’ve got a reasonable understanding of the language you will be able to do some pretty nifty things and have fun doing it!