29 Aug 2018

No Tea in China...

My family and I were lucky enough to spend June in China visiting my wife's family. We spent a week in Hong Kong, and then crossed the border to spend the next 3 weeks in and around Shenzhen. I thought I post a few random thoughts and observations from my trip...

Hong Kong

I found Hong Kong to be a very friendly city, and I enjoy the opportunity of using my very rudimentary Cantonese. Despite being a incredibly busy city of some 7 million people, it is a great place to visit as a family with a young child (though  we passed on visiting the Peak – the tram didn't look pram friendly). Unfortunately, we spent most of our time organising visas for China, so we didn't see as much of Hong Kong as I hoped. My main impressions were the simply fantastic food (we had our best meal in a little local canteen that most tourists wouldn't look twice  at), the excellent metro service (the Octopus card used widely in Hong Kong instead of cash), and a futuristic and incredibly efficient airport. 

https://sites.google.com/site/andyparkhill/home/blog-post-images/MeiAndMatthewinHK.jpg at a public park in Hong Kong

China

We spent most of our time in Shenzhen, a city in Guangdong Province, just to the north of Hong Kong. Most of my wife’s family live here, and we spent a lot of our time visiting her relatives and introducing them to our son Matthew for the first time.

Shenzhen is a massive city of over 16 million inhabitants that has sprung up in just 30 years from what was originally a small fishing town. The city is full of skyscrapers (residents are very proud of having the second tallest building in China) and appears to be continually rebuilding itself every few years. People revisiting the city after a few years often get lost, as the cityscape is unrecognisable from the last time they visited.

Shenzhen skyline at Night

Some random thoughts:

  • Rental bikes are absolutely everywhere, littering the streets, but relatively few people seem to be using them. A lot more people appear to be using battery powered bikes.
  • Unsurprisingly in this new city, there is still a significant wealth disparity, with large numbers of migrant workers from the rural China coming to the city for work.
  • Related to the above, you’ll more commonly speak Mandarin than Cantonese – the people you interact day to day with in Shenzhen (in taxis, in the supermarket) will typically be migrant workers and will not speak Cantonese.
  • Like Hong Kong, Shenzhen has a very efficient public transport system (both the local bus network and the growing metro system). The public transport makes up for the traffic in Shenzhen, which is insane.
  • Living costs in Shenzhen seem to be relatively low (for a Westerner), but high relative to the rest of China.

Whilst in Shenzhen, we paid a visit to the Huaqiangbei, the world famous electronics market. It was an interesting place to visit, but speaking to some of Mei’s family, the market seems to be in decline and employs fewer people than in the past.

As I’m currently working in a UK university, we visited a local university campus in Shenzhen. There are a number of different universities in Shenzhen, notably the Southern University of Science and Technology (see also this article). We visited the campus of the Peking University HSBC Business School, which is an international graduate business school, but we also passed by some of the other universities during our travels. It is striking how large the university campuses are, and how much building is going on at them. Higher education is a major focus of the Chinese government, and it is spending a lot of money to turn them into world class institutions. UK universities will not be able to compete on shiny new buildings alone, something which the UK VCs don’t seem to understand. The only way to compete with Chinese universities will be by delivering world class teaching and research – something that UK VCs are actively harming by treating staff as a hindrance rather than an asset. I expect a lot more Chinese students to opt to study in China in the future as the word standing of their universities continue to increase, along with increasing numbers of foreign students.

Enterance to a Hakka Village near Shenzhen

We also got a chance to visit the nearby smaller cities of Pingshan and Dayawan, as well as spending a couple of nights at a local holiday resort.

Other random thoughts:

  • I noticed a lot more use of people and manual labour (in construction and retail) than would be the case in Europe.
  • There is still significant inequality between the city and rural area. This is unsurprising for a country that has changed, and continues to change, so quickly in recent years..
  • There is a very real exercise and fitness culture in China. There were daily dance and Tai Chi sessions in all the local parks that we visited. The parks were filled in the evenings with people exercising (walking and jogging, playing badminton, attending dance classes) after work.

Exercising in an outdoor gym

  • Driving in China (and particularly Shenzhen) is insane. Defensive driving is essential, as drivers will drive very closely in traffic and will change lanes rapidly without indicating. Whilst the Chinese government is promoting self driving cars, they are a long way from ever appearing on China’s roads.  But given that road accidents kill 700 people a day in China, there will be a greater acceptance of self driven cars in China than in Europe or the US.
  • China is a tough place to visit as a foreigner. If you don’t speak or read the language, there is very little allowance given for you, unlike other countries in Asia. Shenzhen was significantly easier for a foreigner than the smaller cities, but still tough. The major street signs in Shenzhen were in both Chinese and English, which was a life saver. I knew a little Cantonese and very little Mandarin, and couldn’t read any Chinese characters. I’m already looking at learning more Mandarin and learning to read Chinese characters in preparation for my next visit.
  • WeChat is an essential app if you’re visiting China, as well as using a VPN to access websites outside of the Great Firewall of China.
  • Despite trying every coffee and tea shop I came across, I was unable to find a decent cup of black tea with milk. If I hadn’t packed my own teabags, I doubt if I would have survived the whole trip.

China is huge country, and I only spent a brief time there. But I really enjoyed our stay, and I came away impressed with the scale of the country’s ambition and the people I met there. I’m looking forward to our next visit to China already.

28 Aug 2018

Reducing Azure Costs

As part of my study for the Microsoft 70-533 exam, I’ve been interested in finding out how to minimize my costs in Azure. This is particularly important as I’m spinning up a large number of virtual machines and web applications as part of my study for the exam. I’ve gathered the following list of tips on how to reduce your Azure costs.

General

  • Each Azure service is priced differently. Use of the Azure Pricing Calculator to estimate the cost for a specific resource and to identify all the resources/services used to host an application.
  • While is is obvious that different services will have different costs, not everyone is aware that the same service will vary in cost between the different Azure locations. The https://azureprice.net/ site lists the regional pricing for Azure VMs.
  • You’re charged for the resources services you use, so shut down non-production resources when they're not required (like outside working hours). This can be done using either a scheduled script or by using one of the various cloud management applications.
  • Use pre-paid subscriptions to get a discount (even on pay as you go subscriptions) - Microsoft offers discounts of 2.5-5% based on 6 or 12 months pre-payment.
  • If available on your subscription, make use of a spending limit on your Azure account/subscription.
  • Also set up automated email alerts to email you once your account spends more than a set amount.
  • Audit your Azure usage so you know what you are paying for and not using in Azure
    • View your Azure Subscription in the portal
    • Use the free service https://azure-costs.com/ to identify your costs
    • Consider using the Azure Cost Management service offered by Cloudyn, a Microsoft subsidiary.
    • Delete unused resources
  • Use auto-scaling to reduce costs during off hours. These deployment types all support auto-scaling:
    • Cloud Service
    • App Services
    • VM Scale Sets (Including Batch, Service Fabric, Container Service)
  • Scaling could also mean shutting your app down completely. App Services have a feature called AlwaysOn that controls if the app should shut down due to no activity. You could also schedule shutting down your dev/QA servers with something like DevTest Labs. There are also third party services like Park My Cloud.


Virtual Machines

  • Need to shut down and deallocate VMs to stop Azure continuing to reserve the VM's compute resources
    • Use Azure Portal or PowerShell to do this
    • Can also configure the auto shutdown option on the VM
  • Resize over-provisioned VMs
    • Note, the VM will have to be rebooted, so need to avoid when the VM is under peak load
  • Use Azure PaaS features; don’t roll your own on VMs unless you have to.
  • Make use of Dynamic Scaling with VM Scale Sets
  • Use Azure Marketplace VM Images to prevent additional licensing costs (such as for the Windows OS)
  • Use Azure DevTest Labs to automate the start-up and shutdown of VMs, and also implement quotas and policies for VM management
  • Use Reserved Instances to get discounts on the price of VMs - need to reserve for 1 or 3 years
  • Make use of existing Microsoft licences in Azure using the Azure Hybrid Benefit
  • Make use of Azure Site Recovery to back up your VMs
    • Only pay for replication software and the cost of storage


Azure SQL

  • Blob storage offers a cost effective solution to store graphics data. Blob storage of type Table and Queue of 2 GB costs $0.14/month and type block blob costs just $0.05/month. A SQL Database of similar capacity will cost $4.98/month. Hence, use blob storage to store images, videos and text files instead of storing in SQL Database. To reduce the cost and increase the performance, put the large items in the blob storage and store the blob record key in SQL database.
  • If you want to develop, test, or build a proof of concept, then use the freely licensed SQL Server Developer edition. A SQL Server Developer edition VM only incurs charges for the cost of the VM, because there are no associated SQL Server licensing costs.
  • If you want to run a lightweight workload in production (<4 cores, <1 GB memory, <10 GB/database), use the freely licensed SQL Server Express edition. A SQL Server Express edition VM also only incurs charges for the cost of the VM.
  • Use the appropriate edition of SQL Server for other tasks.
  • Use the Pay Per Usage (per second cost of the VM including the SQL Server licence) or the Bring Your Own Licence (BYOL) option as appropriate.
  • Paying the SQL Server licensing per usage is recommended for:
    • Temporary or periodic workloads. For example, an app that needs to support an event for a couple of months every year, or business analysis on Mondays.
    • Workloads with unknown lifetime or scale. For example, an app that may not be required in a few months, or which may require more, or less compute power, depending on demand.
  • Bringing your own SQL licensing through License Mobility is recommended for:
    • Continuous workloads. For example, an app that needs to support business operations 24x7.
    • Workloads with known lifetime and scale. For example, an app that is required for the whole year and which demand has been forecasted.
  • If you have a lot of databases, consider using SQL Elastic pools.
  • Optimize your SQL Database performance


App Services

  • Migrate Apps to Azure App Service, instead of 'Lift and Shift' to a dedicated VM. This is a cheaper option and avoids having to manage a VM.
  • Avoid paying for staging slots on Cloud Services:
    • Delete staging slots as quickly as possible, as these are charged at the same rate as a Production slot
    • Cloud Services cost you money even if the servers are stopped. You have to delete them!
  • Combine web apps with Azure App Service Plans to reduce server count.
  • Stop using Cloud Service Web Roles unless you have to. Use App Services instead - these are also faster to deploy to.
  • Move Worker Roles to Azure Service Fabric or Container Service. This allows you to combine apps and reduce server count and cost.


Miscellaneous

  • Delete the unused VPN gateway and application gateway as they will be charged whether they run inside virtual network or connect to other virtual networks in Azure. Your account will be charged based on the time gateway is provisioned and available.
  • Use Azure Storage for SMB File Shares, instead of a dedicated VM. Again, this is a cheaper option and avoids having to manage a VM.
  • Improve the performance of your application to reduce the amount of cloud resources it uses. Make use of logging and performance monitoring tools/profilers to identify ways of improving your app.
  • Consider using a CDN like CloudFlare for additional caching.
  • Consider the Azure cold storage option for reduced costs
  • Consider using Azure Automation to automatically build and tear down services/resources when required.


References

21 Aug 2018

Microsoft Exam 70-533 - Implementing Microsoft Azure Infrastructure Solutions

As I develop solutions using Azure, and because I want learn more about the platform, I’ve decided to study for the 70-533 Implementing Azure Infrastructure Solutions exam. I have mixed thoughts about certifications, but this one will help in my day-to-day work and seems to be in demand with employers.

I have chosen to do the IT exam (70-533) first, rather than the Developer exam (70-532), as there is more of a focus on PowerShell in 70-533, and I love working with PowerShell. There is also significant overlap between the 2 exams, and I do plan on completing 70-532 at a later date.

As I study, I’ll be posting some blog posts on various Azure related topics. I’ll use this post to list all Azure related posts:

  1. Reducing Azure Costs

10 Aug 2018

Gmail Search and Filters

A quick post about the idiosyncrasies of searching and filtering in Gmail. As a long time Gmail user (since 2008), I’ve made significant use of filters and labels to organise my emails. I also make use of several different domains and email accounts that I manage using Gmail as aliases. However, as I look to move off Gmail and away from Google services, I want to identify exactly who is emailing me with what email address. Specifically, what emails are sent to an alias using the plus operator (andy+newsletter@gmail.com instead of andy@gmail.com)?

Note, andy@gmail.com is NOT my email address, it is simply being used in this post as an example.

To help with this, I have deleted all my existing labels and filters in Gmail, and created new filters that label my mail according to the email address it was sent to, using search terms like to:(*@gmail.com), to:(*@googlemail.com) and to:(*@andyparkhill.co.uk). Note, I since have read and confirmed myself that the wildcard operator “*” doesn’t really work in Gmail search.

By default, the to:(*@gmail.com) filter will capture any email sent to aliases using the plus operator (like andy+newsletter@gmail.com), but it will not capture email sent to aliases that make use of periods (like an.dy@gmail.com).

I had hoped to be able to create a separate label for email sent to the email aliases, but after spending significant time reading up on Gmail search operators and experimenting, I now realise that this is simply not possible. Instead, I can use the existing Gmail label to help search for any emails sent to my Gmail aliases (using the plus operator) using the following search term:

“label:gmail to:-andy@gmail.com” (without quotation marks)

This search will return emails sent to andy+newsletter@gmail.com and andy+hackernews@gmail.com, but not those sent to andy@gmail.com.

Another useful search term I’ve started using is:

“-has:userlabels -in:sent -in:chat -in:draft -in:inbox -from:me” (without quotation marks)

This search will return all emails that are not labelled and that have not been sent by me.

Some useful links: