Organising My Photo Backups

For the last few years, I have been taking regular backups of the various iOS devices around the house. As well as using iTunes, I have taken backups of the photos stored on the device directly. Over time, these photos backups have built up, until I took some time last week to finally organise them.

I wrote the PowerShell script below to process the backup photos by:

• Checking if a photo with the same file hash exists; if it does, I ignore the duplicate file
• I rename the photo to the date and time it was taken, along with the photo resolution, and then move it to the target directory

There are numerous examples of similar scripts out there, so this is nothing special. It could do with being optimised further, and I need to look into assigning a frame resolution for videos. But it took just two hours to put together, and ended up saving me 60 GB of disk space, so it was time well spent.

Setting Row and Column Formats in SharePoint Online with PowerShell

I’m currently spending most of my time working in Office 365, in particular SharePoint Online (along with Microsoft Flow and PowerApps). While the new modern page look is a major improvement on the classic SharePoint on-premise look, customers still want to be able to customize the default look.

To allow customisation of how lists and libraries appear, Microsoft allows you to set view formatting. This uses a JSON object to describe how elements are displayed when a row is loaded in a list view. A useful repository of open source JSON formatting samples is available at https://github.com/SharePoint/sp-dev-list-formatting.

Similarly, you can apply a JSON object to customize how a field (column) is displayed in a list view. Note, neither row or column formatting changes the data in the list item or file; it only changes how it’s displayed to users who browse the list.

While there is a lot of documentation around setting the JSON formatting using the SharePoint Online UI, there is little documentation around using PowerShell to do this. I came across this method of setting the JSON formatting object:

# Get the raw content for the JSON Definition
$listViewFormattingJSON = Get-Content -Raw -Path '.\ViewRowFormat.json';

# Update the List View Formatting Definition
$view = Get-PnPView -List $list -Identity $viewName
$view | Set-PnPView -Values @{CustomFormatter = $listViewFormattingJSON.ToString()}
Invoke-PnPQuery

And similarly, to set the column formatting:

$statusFieldFormattingJSON = Get-Content -Raw -Path '.\StatusColumnFormatting.json';
$statusField = Get-PnPField -List $ListName -Identity 'Status'
$statusField | Set-PnPField -Values @{CustomFormatter = $statusFieldFormattingJSON.ToString()}
Invoke-PnPQuery

These code snippets help to set the customized view below:

Note, I use the SharePoint Patterns and Practices (PnP) PowerShell library to provision and manage my SharePoint Online solutions.

PowerShell Core

I recently read a great post by Paul Cunningham on career advice for IT professionals, and this line stuck with me: ‘Change is a constant’. This really resonated, especially as I belatedly caught up with the recent changes in PowerShell.

I started using PowerShell in 2008, after a more experienced colleague advised me to start spending one afternoon a week learning about new technologies and tools as part of my professional development. This is great advice, by the way, and I would recommend every developer to do it. I also would suggest not asking your manager for permission to do this – if you’re a professional software developer, you need to make time as part of your job to learn new technologies. It is for both for your benefit (professional development) and your company’s (they get a more efficient developer), even if they may not always realise it.

So I started using PowerShell 10 years ago, and I quickly started using it daily, especially as my career started involving more and more SharePoint development, support and administration. However, sometimes I don’t follow my own advice, and I haven’t been keeping up with the major changes happening with PowerShell. It was only when I realised that development had stopped on the AzureRM PowerShell module, and that Microsoft was instead focusing on a new cross platform Az module , that I realised that I had missed a number of major announcements about PowerShell and the direction it was heading. With the release of .Net Core in 2016, Microsoft had started implementing a cross platform version of PowerShell, now known as PowerShell Core 6.0. This was released to general availability back in January 2018. I had missed a major change in an essential developer tool that I use daily.  Change really is constant, and I hadn’t been prepared for this particular one!

The major differences between PowerShell Core and the legacy Windows PowerShell are:

1. PowerShell Core is cross platform and can be run on Windows, Linux and Mac systems.
2. PowerShell Core is open source.
3. Currently, the major breaking changes in PowerShell Core are:
1. PowerShell Workflows are not available.
2. The Out-Gridview command is not available in PowerShell Core
3. PowerShell Core does not support the WMI v1 cmdlets and a large number of other Windows OS specific cmdlets.
4. While Windows PowerShell ships with the ISE editor, PowerShell Core encourages the use of Visual Studio Code.
5. PowerShell core is case-sensitive, as it must now run on Unix operating systems.

As PowerShell Core uses the less feature-rich .NET Core and .NET Standard, it currently only offers a subset of the functionality offered by Windows PowerShell. This will change over time as the Powershell Core framework matures, and as more functionality is developed for it. While Windows PowerShell will continue to be maintained (with bug fixes and security updates), there will be no new functionality added to it.

It is pretty easy to get started using PowerShell Core. It can be installed and run alongside your existing Windows PowerShell. I use Chocolatey to install (almost) everything on my Windows PCs, as it allows me to configure automatic updates. To install PowerShell Core using Chocolatey, run:

choco install powershell-core

Once installed, it is worth configuring the following:

1. Modify your profile. The profile path in PowerShell Core is different to that of Windows PowerShell, and is at C:\Users\{username}\Documents\PowerShell\Microsoft.PowerShell_profile.ps1
2. Modify VS Code to use the PowerShell Core Integrated Terminal.
3. Install the Windows Compatibility module to allow PowerShell Core to invoke commands that are currently only available in Windows PowerShell. This will allow you to run existing PowerShell scripts in PowerShell Core without any changes.

As I’m spending a lot of time in Azure, the first new PowerShell Core module I installed was the new Az module, which replaces the older AzureRM modules. Once the module is installed, to ensure compatibility with existing AzureRm scripts you should run the Enable-AzureRmAlias cmdlet, to enable aliases for your existing scripts for the current session only.

Calling Executable Files in PowerShell

I was called out on my approach on this in work today, so I thought I would share my thoughts on this topic.  The actual executable concerned was STSADM command line tool for SharePoint 2010 (yes, STSADM is still useful in SharePoint 2010, for example in enabling self-service site creation). Unfortunately, I don’t have STSADM installed on the machine I’m writing this blog post on, so I use the example of calling Internet Explorer instead. The principal is the same in both cases.

To call Internet Explorer from the command line, you would use the following command:

C:\Program Files\Internet Explorer\iexplore.exe

But if you try and run the above command in PowerShell, you will get the following error:

To correct the error, we need to enclose the path to the iexplore.exe file in quotes:

“C:\Program Files\Internet Explorer\iexplore.exe”

But this will not actually result in IE being called; this is because PowerShell treats the input as a string, and will simply write the text enclosed in the quotes to the console.  To run the string as a script whose path is enclosed in quotes, you preface the string with the call operator (ampersand):

& “C:\Program Files\Internet Explorer\iexplore.exe”

Alternatively, we can use the Invoke-Item cmdlet:

Invoke-Item “C:\Program Files\Internet Explorer\iexplore.exe”

And both approaches will successfully start an instance of Internet Explorer.  Alternatively, you could break the statement into two:

CD “C:\Program Files\Internet Explorer\”
iexplore.exe

Which would also work.  Note, if you needed to return to the original directory from which the PowerShell script was called, you could do so using the following code:

Push-Location “C:\Program Files\Internet Explorer\”
iexplore.exe
Pop-Location

The cmdlet Push-Location is virtually identical to the CD command; however, it saves your previous location before moving to a new one.

So, can we now specify at the PowerShell prompt the URL that IE opens with? The command line switch information for IE tells us that we can, by adding an additional parameter:

& “C:\Program Files\Internet Explorer\iexplore.exe” http://www.bbc.co.uk/news/

And again, this successfully works.  Can we now call IE with several parameters?

& “C:\Program Files\Internet Explorer\iexplore.exe” http://www.bbc.co.uk/news/ –private

And this PowerShell statement will call Internet Explorer so that it opens at the BBC News site in p0rn InPrivate mode.  So that we can see that to call STSADM from PowerShell, we could use the following command:

& “C:\Program Files\Common Files\Microsoft Shared\Web server extensions\14\bin\stsadm.exe” -o enablessc -url http://myserver –requiresecondarycontact

And this would work.  Except, of course, that no self respecting software developer would leave should a horribly long line of code in a script.  They would refactor this line of code by using an alias for the STSADM executable, and so avoid having to repeatedly use a hard-coded literal for the executable path.  They would also realise that the location of the common programs varies according to how Windows itself was installed on the target computer, and make use of the %CommonProgramFiles% environment variable:

set-alias STSADM "${env:commonprogramfiles}\Microsoft Shared\Web Server Extensions\14\BIN\STSADM.EXE"
$serverUrl = http://myserver
…
STSADM –o enablessc –url $serverUrl –requiresecondarycontact

The above code is now more easily read and maintained.  If you want to investigate further how PowerShell parses quotes, etc., check out this StackOverFlow question, and Keith Hill’s answer:

Sending Email Using PowerShell

I’ve just been called out on the fact that I didn’t complete my mini-blog series on sending emails using PowerShell.  So, to finally conclude those articles, an example of using the Send-MailMessage cmdlet that was introduced in PowerShell V2.0.

Send-MailMessage –From "recipient@target.com" –To "sender@source.com" –Subject "Test" –Body "A test of the Send-MailMessage cmdlet"  -Attachments "c:\Attachment.xls" –SmtpServer smtpServer.com

In order to send a email with multiple attachments, you need to pass an array with the full file paths of all the attachments to the -Attachments parameter. This is easily done in PowerShell by simply piping the contents to the Send-MailMessage cmdlet.

Get-ChildItem "C:\Folder" -Include *.txt | Where {-NOT $_.PSIsContainer} | foreach {$_.fullname} |
Send-MailMessage -From "recipient@target.com" -To "sender@source.com"  -subject "Test" -smtpServer smtpServer.com

Or to send each attachment separately:

foreach ($attachment in $attachments)
{
	Send-MailMessage -From "recipient@target.com" -To "sender@source.com"  -subject "Test" -smtpServer smtpServer.com -attachments $attachment
}

PowerShell and Arrays

I came across a strange feature of PowerShell recently.  When dealing with an array that held a single item, I was very surprised to see that the count of items was over 1000.  To illustrate the problem, consider the code below:

		
$files = Get-ChildItem C:\Temp
# Single file present in target folder

Write-Host "Total number of files: $files.Length"
# $files.Length returns value in 1000s
		
for ($i = 0; $i -lt $files.Length; $i++)
{
	$file = $files[$i].Name
	Write-Host "Processing $file"
}

When the code was run for the case when there was single file in the target directory, the count came back in the 1000s. The script then raised an exception for each of the 1000s of times it attempted to reference files[i], when no such array index existed.

It took some digging to realise that what was actually happening was that when PowerShell returns a collection with one or zero items in the collection, it unpacks the array and returns the item (or $null), and not an array.  So in the instance above, by calling $files.Length, the script was returning the length of the file object in bytes, and not the size of the array. 

To correct this behaviour, simply wrap the call generating the array with @(), as shown below:

		
$files = @(Get-ChildItem C:\Temp)

This will have no effect if the call returns an array with more than one item (i.e. you will not have an array wrapped in an array). See this StackTack answer from Keith Hill for a more detailed response:

Also, just to note that the code for this post was highlighted using the Google Code Prettify JavaScript library.

Send Outlook Email Via PowerShell

Just to complete the topic of sending emails via PowerShell, [Lies, I still have to blog about the Send-MailMessage cmdlet.] I thought I would include a quick look at sending emails from Outlook via PowerShell.  Again, this script could be replaced by using the PowerShell cmdlet, Send-MailMessage. 

The script below is based on a script submitted to TechNet by Kent Finkle:

   1:  param 

   2:  (        

   3:      [string]$email = $(read-host "Enter a recipient email"),

   4:      [string]$subject = $(read-host "Enter the subject header"), 

   5:      [string]$body = $(read-host "Enter the email text (optional)")

   6:  )   

   7:   

   8:  # Functions

   9:   

  10:  function Send-Email

  11:  (

  12:      [string]$recipientEmail = $(Throw "At least one recipient email is required!"), 

  13:      [string]$subject = $(Throw "An email subject header is required!"), 

  14:      [string]$body

  15:  )

  16:  {

  17:      $outlook = New-Object -comObject  Outlook.Application 

  18:      $mail = $outlook.CreateItem(0) 

  19:      $mail.Recipients.Add($recipientEmail) 

  20:      $mail.Subject = $subject 

  21:      $mail.Body = $body   # For HTML encoded emails 

  22:      # $mail.HTMLBody = "<HTML><HEAD>Text<B>BOLD</B>  <span style='color:#E36C0A'>Color Text</span></HEAD></HTML>"   

  23:      # To send an attachment 

  24:      # $mail.Attachments.Add("C:\Temp\Test.txt")    

  25:      $mail.Send() 

  26:      Write-Host "Email sent!"

  27:  }   

  28:   

  29:  # Main Script Body

  30:   

  31:  Write-Host "Starting Send-MailViaOutlook Script."   

  32:  # Send email using Outlook

  33:  Send-Email -recipientEmail $email -subject $subject -body $body     

  34:  Write-Host "Closing Send-MailViaOutlook Script."   

  35:   

  36:  # End of Script Body

Download the script from here.  If you have any comments, I would appreciate all feedback.  Thanks.

Also, just to note that the code for this post was highlighted (badly!) using the Neat Highlighter website. Unfortunately, the code snippet produced by the Neat Highlighter website was so poor that I couldn’t use it, and I have updated this post to use a snippet generated by the Insert Code for Windows Live Writer plugin. I also tried to use the Code Formatter Plugin, as recommended by Scott Hanselman, but it constantly froze Windows Live Writer. Also, I discovered how to use the <strike> HTML tag!

Sending Gmail via PowerShell

[Update: I have updated this blog to display the StackOverflow question using the new StackTack jQuery plugin.]

In a previous post, I looked at reducing the number of mouse clicks and key strokes required to send an email and other common tasks using Launchy.  The solution involved using Outlook, as this was for my work PC.  I started thinking about doing the same on my home laptop, where I don’t have Outlook installed.  I remembered a blog article from Scott Hansleman on using the command line email tool Blat, and thought I could do something similar using a PowerShell script send email via Gmail.

Just to note, the script below is only valid for PowerShell V1; in PowerShell V2, there is a cmdlet available to do this, Send-MailMessage. I’ll look at this using this cmdlet in my next blog post.

A quick search of StackOverflow found the following:

My script is based on the answer submitted by user Christian:

param 
(        
    [string]$email = $(read-host "Enter a recipient email"),
    [string]$subject = $(read-host "Enter the subject header"), 
    [string]$body = $(read-host "Enter the email text (optional)")
)

# ==========================================================================
#     Constants
# ==========================================================================

set-variable -name EmailAddress -value "testuser@gmail.com" -option constant
set-variable -name UserName -value "testuser" -option constant
set-variable -name PassWord -value "Password123" -option constant

# ==========================================================================
#     Functions
# ==========================================================================

function Send-Email
(
    [string]$recipientEmail = $(Throw "At least one recipient email is required!"), 
    [string]$subject = $(Throw "An email subject header is required!"), 
    [string]$body
)
{
    $EmailFrom = $EmailAddress
    $EmailTo =  $recipientEmail
    $Subject = $subject
    $Body = $body
    $SMTPServer = "smtp.gmail.com" 
    $SMTPClient = New-Object Net.Mail.SmtpClient($SmtpServer, 587) 
    $SMTPClient.EnableSsl = $true 
    $SMTPClient.Credentials = New-Object System.Net.NetworkCredential($UserName, $PassWord); 
    $SMTPClient.Send($EmailFrom, $EmailTo, $Subject, $Body)

    Write-Host "Email sent!"
}

# ==========================================================================
#    Main Script Body
# ==========================================================================

Write-Host "Starting Send-Gmail Script."

# Send email using Gmail
Send-Email -recipientEmail $email -subject $subject -body $body

Write-Host "Closing Send-Gmail Script."

# ==========================================================================
#    End of Script Body
# ==========================================================================

Using the script above, and the previously mentioned way of calling PowerShell scripts from Launchy, I can quickly generate emails with a minimal number of keystrokes.

Download the script from here.  If you have any comments, I would appreciate all feedback.  Thanks.

Also, just to note that the code for this post was highlighted using the BlogTrog website.

Fevered Coding

I am suffering a nasty chest infection, and it has been tough getting to bed over the last few days.  As a result, I have been staying up late, coding on a few random scripts. 

The result of last night’s fevered coding was a PowerShell script to return the ratings for various TV shows on IMDB.  This is my first attempt at the script, and it makes use of the IMDB web service provided by Dean Clatworthy.

The script consists of two main parts.  The first is a call to the web service, and returning the details for the specified TV show, using an instance of the .NET WebClient.

  1: function Get-ShowDetails
  2: (
  3:   [string]$showName = $(Throw "TV show name is required!")
  4: )
  5: {  
  6:   $webClient = new-object System.Net.WebClient
  7:   $webClient.Headers.Add("user-agent", "PowerShell Script")
  8:   $queryString = $showName.Trim().Replace(" ", "+")
  9:   $query = [string]::Format("{0}={1}", $queryUrl, $queryString)   
 10:   $showDetails = $webClient.DownloadString($query)
 11:   return $showDetails
 12: }

The second part of the script extracts the rating details for the show from the plain text returned by the web service, using a regular expression:

  1: function Extract-RatingDetails
  2: (
  3:   [string]$showName = $(Throw "TV show name is required!"),
  4:   [string]$showDetails = $(Throw "TV show details are required!")
  5: )
  6: {  
  7:   $regexPattern = '"rating":"([a-z\\\/\.0-9]+)","votes":([0-9\"]+)'
  8:   $regexMatcher = [regex] "$regexPattern"
  9:   $matches = $regexMatcher.Matches($showDetails)
 10:   $showInfo = $null  
 11:   
 12:   foreach ($match in $matches)
 13:   {
 14:     if ($match.Success)
 15:     {
 16:       # To return all of the matched expression, use $match.Groups[0].Value
 17:       $rating = $match.Groups[1].Value
 18:       
 19:       if($rating -eq "n\/a" )
 20:       {
 21:         $rating = 0;
 22:       }
 23:       
 24:       $votes = $match.Groups[2].Value.Replace('"', '')      
 25:       $showInfo = New-Object PSObject
 26:       $showInfo | Add-Member NoteProperty "ShowTitle" $showName
 27:       $showInfo | Add-Member NoteProperty "Rating" $rating
 28:       $showInfo | Add-Member NoteProperty "Votes" $votes      
 29:       break;
 30:     }
 31:   }
 32:   
 33:   return $showInfo
 34: }

This isn’t a finished script, though it does work.  There is a limit to the number of calls that can be made to the web service (30 per hour), which means that I will have to revisit the script and instead of calling the web service, scrape the details of each TV series from the IMDB website. 

Download the current script from here.  If you have any comments, I would appreciate all feedback.  Thanks.

Also, just to note that the code for this post was highlighted using the Source Code Formatter for Windows Live Writer.