Reducing Azure Costs

As part of my study for the Microsoft 70-533 exam, I’ve been interested in finding out how to minimize my costs in Azure. This is particularly important as I’m spinning up a large number of virtual machines and web applications as part of my study for the exam. I’ve gathered the following list of tips on how to reduce your Azure costs.

General

• Each Azure service is priced differently. Use of the Azure Pricing Calculator to estimate the cost for a specific resource and to identify all the resources/services used to host an application.
• While is is obvious that different services will have different costs, not everyone is aware that the same service will vary in cost between the different Azure locations. The https://azureprice.net/ site lists the regional pricing for Azure VMs.
• You’re charged for the resources services you use, so shut down non-production resources when they're not required (like outside working hours). This can be done using either a scheduled script or by using one of the various cloud management applications.
• Use pre-paid subscriptions to get a discount (even on pay as you go subscriptions) - Microsoft offers discounts of 2.5-5% based on 6 or 12 months pre-payment.
• If available on your subscription, make use of a spending limit on your Azure account/subscription.
• Also set up automated email alerts to email you once your account spends more than a set amount.
• Audit your Azure usage so you know what you are paying for and not using in Azure
• View your Azure Subscription in the portal
• Use the free service https://azure-costs.com/ to identify your costs
• Consider using the Azure Cost Management service offered by Cloudyn, a Microsoft subsidiary.
• Delete unused resources
• Use auto-scaling to reduce costs during off hours. These deployment types all support auto-scaling:
• Cloud Service
• App Services
• VM Scale Sets (Including Batch, Service Fabric, Container Service)
• Scaling could also mean shutting your app down completely. App Services have a feature called AlwaysOn that controls if the app should shut down due to no activity. You could also schedule shutting down your dev/QA servers with something like DevTest Labs. There are also third party services like Park My Cloud.

Virtual Machines

• Need to shut down and deallocate VMs to stop Azure continuing to reserve the VM's compute resources
  • Use Azure Portal or PowerShell to do this
  • Can also configure the auto shutdown option on the VM
• Resize over-provisioned VMs
  • Note, the VM will have to be rebooted, so need to avoid when the VM is under peak load
• Use Azure PaaS features; don’t roll your own on VMs unless you have to.
• Make use of Dynamic Scaling with VM Scale Sets
• Use Azure Marketplace VM Images to prevent additional licensing costs (such as for the Windows OS)
• Use Azure DevTest Labs to automate the start-up and shutdown of VMs, and also implement quotas and policies for VM management
• Use Reserved Instances to get discounts on the price of VMs - need to reserve for 1 or 3 years
• Make use of existing Microsoft licences in Azure using the Azure Hybrid Benefit
• Make use of Azure Site Recovery to back up your VMs
  • Only pay for replication software and the cost of storage

Azure SQL

• Blob storage offers a cost effective solution to store graphics data. Blob storage of type Table and Queue of 2 GB costs $0.14/month and type block blob costs just $0.05/month. A SQL Database of similar capacity will cost $4.98/month. Hence, use blob storage to store images, videos and text files instead of storing in SQL Database. To reduce the cost and increase the performance, put the large items in the blob storage and store the blob record key in SQL database.
• If you want to develop, test, or build a proof of concept, then use the freely licensed SQL Server Developer edition. A SQL Server Developer edition VM only incurs charges for the cost of the VM, because there are no associated SQL Server licensing costs.
• If you want to run a lightweight workload in production (<4 cores, <1 GB memory, <10 GB/database), use the freely licensed SQL Server Express edition. A SQL Server Express edition VM also only incurs charges for the cost of the VM.
• Use the appropriate edition of SQL Server for other tasks.
• Use the Pay Per Usage (per second cost of the VM including the SQL Server licence) or the Bring Your Own Licence (BYOL) option as appropriate.
• Paying the SQL Server licensing per usage is recommended for:
  • Temporary or periodic workloads. For example, an app that needs to support an event for a couple of months every year, or business analysis on Mondays.
  • Workloads with unknown lifetime or scale. For example, an app that may not be required in a few months, or which may require more, or less compute power, depending on demand.
• Bringing your own SQL licensing through License Mobility is recommended for:
  • Continuous workloads. For example, an app that needs to support business operations 24x7.
  • Workloads with known lifetime and scale. For example, an app that is required for the whole year and which demand has been forecasted.
• If you have a lot of databases, consider using SQL Elastic pools.
• Optimize your SQL Database performance

App Services

• Migrate Apps to Azure App Service, instead of 'Lift and Shift' to a dedicated VM. This is a cheaper option and avoids having to manage a VM.
• Avoid paying for staging slots on Cloud Services:
  • Delete staging slots as quickly as possible, as these are charged at the same rate as a Production slot
  • Cloud Services cost you money even if the servers are stopped. You have to delete them!
• Combine web apps with Azure App Service Plans to reduce server count.
• Stop using Cloud Service Web Roles unless you have to. Use App Services instead - these are also faster to deploy to.
• Move Worker Roles to Azure Service Fabric or Container Service. This allows you to combine apps and reduce server count and cost.

Miscellaneous

• Delete the unused VPN gateway and application gateway as they will be charged whether they run inside virtual network or connect to other virtual networks in Azure. Your account will be charged based on the time gateway is provisioned and available.
• Use Azure Storage for SMB File Shares, instead of a dedicated VM. Again, this is a cheaper option and avoids having to manage a VM.
• Improve the performance of your application to reduce the amount of cloud resources it uses. Make use of logging and performance monitoring tools/profilers to identify ways of improving your app.
• Consider using a CDN like CloudFlare for additional caching.
• Consider the Azure cold storage option for reduced costs
• Consider using Azure Automation to automatically build and tear down services/resources when required.

References

• https://8kmiles.com/blog/7-tips-to-save-costs-in-azure-cloud-management/
• https://buildazure.com/2017/04/08/top-10-tricks-to-save-money-with-azure-virtual-machines/
• https://www.swc.com/blog/cloud/five-ways-save-money-on-azure-vms
• https://docs.microsoft.com/en-us/azure/virtual-machines/windows/sql/virtual-machines-windows-sql-server-pricing-guidance
• https://www.cloudcomputing-news.net/news/2018/may/04/microsoft-azure-pricing-101-tips-and-tricks/

How to Fix: ‘The trust relationship between this workstation and the primary domain failed’

I come across this error several times a year, typically when I start up a Virtual Machine that hasn’t been used in a while. Many people will recommend leaving and then re-joining the AD to resolve this problem – this is a terrible idea. I came across this solution a couple of years ago, and after having to search it for again today, I decided to post it here so I can find it more easily in the future.

To fix the error, reset the computer password using the netdom command line utility:

netdom.exe resetpwd /s:<server> /ud:<user> /pd:<password>

where:

• <server> is a domain controller in the joined domain. You can get the domanin controller name using this command: nltest /dclist:domainname
• <user>is in DOMAIN\User format, and has the required rights to change the computer password

I typically log on to the affected virtual machine using a local administrator account. I run the command above using  an elevated command prompt. Once the command succeeds, simply reboot the machine, and you will then be able to log on using a domain account as normal.

Moving to GitLab

I’ve recently moved my repositories from GitHub to GitLab. I thought I share a quick post on why I did this and how to go about it.

Why Move?

GitLab has a number of advantages over GitHub for source code hosting:

1. Free private repositories – GitLab is financed by their Enterprise tiers, and have a free tier for personal projects and small teams. GitHub is free only for public repositories.
2. GitLab has built-in Continuous Integration/Delivery options built-in, and has a number of DevOps features available to integrate with your specific workflow.
3. GitLab has a wider range of options for creating project specific sites (using GitLab Pages), such as allowing you to use custom domains and SSL certificates for each project.
4. GitLab, unlike GitHub, has an open source community edition (with an MIT licence), allowing you to setup your own GitLab server to self-host.
5. GitLab has pretty much all the features that you expect from GitHub (Wikis, Markdown-based readmes, issue tracking). An important consideration for me was that GitLab, like GitHub, also offers Two Factor Authentication (2FA).

The disadvantages of moving to GitLab are:

1. As GitHub has been around longer, network effects mean that the majority of developers and projects are already using it. As of 2017, GitHub had some 26 million user and 67 million repositories.
2. Occasionally you can have slow UI performance.
3. Both offer integration with a number of third party services (such as Trello, Slack, Jenkins) but GitHub currently offers more integration options than GitLab.

How to Move

GitLab has made made migrating from GitHub very easy. Simply create a new project, and on the new project page, select the ‘Import project’ tab. From here, you can select the GitHub import option. You will then be asked to sign in to GitHub and to authorize GitLab to access your GitHub repositories. On doing this, you are taken to the GitHub Importer page, which lists your projects on GitHub. You can then select the projects to import to GitLab individually, or simply import them all:

Note, the import preserves the repositories commit history. Once the projects are successfully imported, you can then decide whether to delete the source projects on GitHub. I deleted all of my private repositories on GitHub, and maintained only my public projects.

As GitHub is the more popular site for hosting source code, I wanted to be able to keep hosting and updating my public projects there easily. I had thought initially to add GitHub as a second remote to the new GitLab repository, so I could update both GitHub and GitLab repositories from the command line. However, it turns out that GitLab offers the Repository Mirroring feature. From the GitLab documentation:

‘There are two kinds of repository mirroring features supported by GitLab:
push and pull. The push method mirrors the repository in GitLab
to another location, whereas the pull method mirrors an external repository
in one in GitLab.’

By configuring the public repositories to push to the corresponding GitHub repository, I can simply commit my code to the GitLab repo as normal, and the commits are automatically reflected in the mapped GitHub project.

Once the push mirroring was configured, I found that I could commit to GitLab and the code changes would be pushed to GitHub in a matter of seconds!

Migrating to GitLab was a very simple process, and so far, I’m very happy that I made the move.

Forgotten Your Restriction PIN for IPhone?

Recently, I forgot the restrictions PIN on my iPhone (fat thumbs when initially setting it). I eventually recovered it by using the very useful pinfinder utility. This console program, written in Go, was kindly developed and open sourced by Gareth Watts. His utility recovers the restrictions (parental) passcode on iPhones, iPads and iPod touches from an (unencrypted) iTunes backup.

I’m blogging about this to help anyone with the same problem, and also to recommend pinfinder, so people use it rather than one of the rather dubious other commercial products that are available.

Download the latest release of pinfinder from here:

https://github.com/gwatts/pinfinder/releases

A Base .NET Console Application

I often use use console applications to put together proof of concepts and demos.  So I was very interested in this base console application from John Atten. However, while John’s console template was built to be interactive, I wanted a simpler base console that I could use as a starting point for any project. 

The source code for the application can be found on GitHub. 

At the minute, the solution is nothing more than a skeleton application with logging, error handling and strongly typed configuration settings provided.  It also makes use of a command line parsing library to allow easy addition of new commands. 

This is just an initial draft of the base console application. As I’m working on a couple of projects that will make use of this base console application, I’ll update the solution with any improvements I can think of in the next few weeks. I'm also happy to take pull requests, suggestions, and ideas for ways it can be extended.

Also, when developing the solution, I came across a minor issue with the BizArk toolkit, used for the command line parsing. I submitted a patch for the issue (that can be seen here), following these instructions from Scott Hanselman. My first patch to an open source solution!

Cross Browser Testing

I have just finished a major refactoring of a legacy website.  I thought it would be useful to share some of the lessons I learnt.  One of the main lessons I learnt was to make the best use of the tools available.  In particular, I spent a lot of time testing for cross browser compatibility (this legacy site was originally developed in 2007/08), and I wanted to discover the best way of quickly uncovering any browser rendering issues. 
There are several different ways of testing websites for cross browser compatibility:

• Installing multiple versions of the various browsers on a local PC to allow for testing (belts and braces).  This can be difficult, as some browsers don’t allow for different versions being installed on the same system (Cough *Internet Explorer* Cough). Normally, you end up using multiple virtual images with the various target browsers installed, which can be a pain to manage.
  
• Use a web-based screenshot service in order to view how a site renders on various combinations of browsers and operating systems. A multitude of these services have sprung up recently, making use of various cloud-based virtual environments.  Several of these sites allow you to use a browser on a remote machine and test the functionality, as well checking the actual page rendering.
  • CrossBrowserTesting – A commercial service that offers both automated screenshots and access to browsers in remote machine sessions to interactively test your website. Covers all major browsers and operating systems, including all mobile smartphone devices.

• BrowserCam – Another commercial service, again offering both automated screenshots and remote access to various browsers. Covers all major browsers and operating systems, including all mobile smartphone devices.
  
• Adobe BrowserLab – An excellent online service that allows you to view sites side by side and even to set different browser renderings over the top of one another (thought this onion skin mode isn’t that useful in practice!).  The service is fast and allows you to test using Safari (5.1), Internet Explorer (6 – 9), Firefox (5, 7 on both OS X and Windows) and Chrome (13 and 14).  The service is currently free until April 2012, when it will become a paid service.

• Spoon – A free virtualization service that allows you make use of all the major browsers except Internet Explorer.
• Desktop applications typically bundle multiple browser installations, allowing you to switch browsers when viewing your site.
  • Lunascape Browser - This browser comes with all three major browser engines, allowing it to mimic the behaviour of all the major browsers (IE, Firefox, Chrome, Safari).  The browser is free to download.
    
  • Microsoft Expression Web SuperPreview – A free tool to allow to test all versions of IE locally, and also to test the other browsers remotely. This tool is free, and is normally found as part of the Expression IDE.

• Browser developer tools and plugins.  There are various browser tools and plugins to allow developers to check how web pages will display in other browsers.
  • IE Tab for Firefox
    
  • Web Developer Toolbar for Firefox
    
  • F12 Developer Tools in Internet Explorer (v8+).

Of the above methods, I would recommend Microsoft Expression Web SuperPreview as the best way of checking how your site renders in the main browsers, because it is both free and fully featured   I found Adobe BrowserLab to be the best online solution, but I wasn’t testing against mobile devices.  The two commercial offerings may be more relevant if you are developing against mobile devices.  I intend to look at cross browser testing on mobile devices in a future post.