Securing your IPv6-only docker server

It is important to ensure your IPv6-only docker server is secure.

First configure your firewall to allow secure shell (SSH), port 22, so that you can maintain your remote connection.

Then turn on your firewall with default deny incoming and default deny routing rules.

This ensures your server is secure-by-default, and only then should you allow routing to the specific containers and ports that you want to expose.

My server runs Ubuntu, so these instructions are based on the Uncompliciated Firewall (UFW), but similar considerations apply to other platforms

Continue reading Securing your IPv6-only docker server

The end of 3G for IoT

3G (3rd generation mobile technology) networks for the major telecommunication companies are due to shut down over the next few years. This includes Telstra, whose network is now in the sunset phase and due to close in June 2024.

This will mean the end of 3G for Internet of Things deployments, and they will need to migrate to either LPWAN (Low-Power, Wide-Area Networks) or new generation cellular mobile, depending on the use case.

As pointed out in this article on Why you need to migrate your devices now! that does not give a lot of time. If you have 15,000 devices in the field you need to be replacing 30 devices per day — if you start tomorrow; more if you take long to commence your project.

The are three main options for migration, in two categories:

  • LPWAN
    • NB-IoT (Narrow-Band Internet-of-Things)
    • Cat-M1 (Category M1), also known as LTE-M (Long Term Evolution, Category M)
  • Cellular Mobile
    • 4G LTE (4th Generation) mobile

This post will explore those options in a bit more detail, as well as what other alternatives there might be. 5G NR (5th Generation New Radio) does not yet have wide enough coverage to be a viable option for IoT in most cases.

If this seems a bit overwhelming, given the short time frames and what you need to do, then you can also approach our consulting services, Telstra Purple, for advice and help.

Continue reading The end of 3G for IoT

Running an IPv6-only host — redux

I have previously blogged about why you should consider IPv6 only hosting and setting up Apps on Kubernetes IPv6 to run my WordPress blog.

Kubernetes is not really designed for a single server (but is great for scaling and enterprise system), and although it was good experience learning how to set it up on IPv6, the overhead was too much and I eventually ended up with a crashed blog.

I'm still running IPv6 only, but with a much simpler set up.

This consists of docker, configured to run with IPv6, with docker-compose to run the different components and systems.

If you are planning on setting up your own server, read my notes on Securing your IPv6-only docker server before starting.

On my server there are currently three instances of WordPress for different websites, and 3 corresponding databases, as well as a Matrix Synapse server and plugins.

Read on for my notes on initial setup of the server with IPv6 and connectivity testing, including addressing schemes, docker configuration, IPv6 network address translation, and the Network Discovery Protocol Proxy Daemon.

Continue reading Running an IPv6-only host — redux

Unboxing the Dragino LPS8 LoRaWAN gateway

I recently got a Dragino LPS8 LoRaWAN gateway and set it up on my network. The LPS stands for LoRaWAN Pico Station.

The open source gateway runs a variant of OpenWRT and the latest version supports a range of LoRaWAN features including Basic Station. You can use it for a private network or set it up with a community as I did for The Things Network (TTN).

Unboxing the Dragino LPS8 LoRaWAN gateway

Read on for details of how easy it was to set it up securely.

Continue reading Unboxing the Dragino LPS8 LoRaWAN gateway

Modern distributed tracing with dotnet

For any modern dotnet system, distributed tracing is already built in to the default web client, server, and other operations.

You can see this with a basic example, by configuring your logging to display the correlation identifier. Many logging systems, such as Elasticsearch, display correlation by default. The identifiers can also be passed across messaging, such as Azure Message Bus.

Logging has always been a useful way to understand what is happening within a system and for troubleshooting. However, modern systems are very complex, with multiple layers and tiers, including third party systems.

Trying to understand what has happened when a back end service log reports an error, and then correlating that to various messages and front end actions that triggered it requires some kind of correlation identifier.

This problem has existed ever since we have had commercial websites, for over 20 years of my career, with various custom solutions.

Finally, in 2020, a standard was created for the format of the correlation identifier, and how to pass the values: W3C Trace Context. In a very short amount of time all major technology providers have implemented solutions.

The examples below show how this is already implemented in modern dotnet.

Continue reading Modern distributed tracing with dotnet

Code First Azure Digital Twins — a first look

Telstra has a large Internet of Things portfolio, with Digital Twins one of the focus areas for Telstra Purple professional services. All major providers are supported, including Azure Digital Twins.

The team recently took some core bits out of project they are working on with code first Azure Digital Twins and have released it as an open source library, so I thought I would share an initial look at the project.

Why code first? Using a code first approach can make accessing Digital Twins easier for developers. They can use their native programming language and tools to develop their models, without having to learn the intricacies of DTDL (Digital Twins Definition Language) or the REST APIs for interacting with Azure Digital Twins.

The library can be found at https://github.com/telstra/DigitalTwins-CodeFirst-dotnet

Continue reading Code First Azure Digital Twins — a first look

Azure CLI vs PowerShell vs ARM vs Bicep

A key component of DevSecOps is infrastructure-as-code, and if you are using Azure there are multiple ways to specify what you want.

Microsoft provides Azure PowerShell, the Azure CLI, as well as both Azure Resource Manager (ARM) and the newer Bicep templates. There are also third party (and cross-cloud) solutions such as Terraform and Pulumi.

In the past I have been leaning towards Azure CLI, as I found ARM templates a bit cumbersome, plus my previous experience with migrations vs desired state for database deployments. With Bicep being promoted as a lighter weight alternative I thought I would compare the Microsoft alternatives.

Having now revisited the options, I still prefer scripting, but think I will switch more to PowerShell, particularly as it makes it easier to follow to the tagging, and naming, guidelines.

My recommendations:

  • For incremental development or changing environments, use Azure PowerShell scripts. They allow easy manipulation of parameters, and a migration/scripted approach can handle changes that a desired state/template approach can not.
    • If you are already heavily invested in an alternative scripting system, e.g. Bash, then Azure CLI would be easier to use.
  • If you have relatively stable infrastructure, such as a preset development environment or sample/demo code, that you want to repeatedly tear down and then recreate the same, then Bicep offers a nicer syntax than raw ARM templates. The deployments are viewable in the Azure portal, but templates do have some limitations compared to scripting.
  • In either case, follow the Azure Cloud Adoption Framework naming guidelines, allowing for unique global resources, as well as the associated tagging guidelines.

Example code is available on Github at https://github.com/sgryphon/azure-deployment-examples

Continue reading Azure CLI vs PowerShell vs ARM vs Bicep

Evaluating blockchain networks

What is blockchain? What makes a good blockchain? Is blockchain just a buzzword? Is a blockchain really trustless?

Let's start with the last one. The word trustless is often used to describe blockchain, however it is not really trustless — it would be better described as decentralised trust or systemic trust (trust in the system).

Rather than trust an individual (person or organisation) the trust is provided by the very system itself.

You do need to trust the transaction processors (aka miners) for the chain, and also the programmers that wrote the software being used. However you don't need to trust the individual processors or software, only that the majority are truthful. A characteristic of blockchain is that an individual bad actor, controlling only a small portion of the network, cannot take the network down.

A blockchain network can increase systemic trust through:

  • A large number of processors (e.g. public chain)
  • Multiple clients (i.e. multiple development teams)
  • Open source (allow visibility and the option to reject changes and go a different direction)

Only two major blockchains rate highly on these trust criteria: Bitcoin (BTC) and Ethereum (ETH). There are strong reasons why these have strong support.

Some of the others are trying, but not yet there, and worryingly some of the type cryptocurrencies are not decentralised at all, but actually controlled by a single central organisation.

Continue reading Evaluating blockchain networks

Crashed blog… now restored

So, I pushed the single-server Kubernetes cluster that I was running my blog on a little too far, and it crashed into a bit of a heap. The pods running the different sites, including this blog, failed, and the underlying database got corrupted.

It has been down for a few weeks now. Initially I thought it was just a server issue and rebooted. When it didn't come up, I did little bits of investigation over the following weeks, just a few hours at a time, to figure out the issue.

I managed to work out how to restore the database and get it working, but the server was not stable. It would quickly crash, and trying to activate more than one site would just cause problems.

Kubernetes is quite complicated, and there is a lot of overhead for a single server. It was still a good exercise to understand the complexities of deploying Kubernetes on IPv6.

Now, deploying multiple services via containers is still a good approach, with Kubernetes simply a way to orchestrate, and manage, a large number of containers. So, I can pretty much just run the same containers, just directly (instead of inside Kubernetes).

As you can see, from this blog entry, my services are now back up and running.

There was still the complexity of running on IPv6 only, which I should probably write up in more detail, but for now a lot of it was based on an article by Stefan Kleeschulte, https://medium.com/@skleeschulte/how-to-enable-ipv6-for-docker-containers-on-ubuntu-18-04-c68394a219a2

App Insights trace correlation

Application Insights is the application performance monitoring feature of Azure Monitor, and can be used to monitor deployed applications both in the cloud and on premises. App Insights supports W3C Trace Context standard headers to correlate tracing information across different components.

The features of App Insights, and Azure Monitor, are quite broad, whereas developers may want in some cases to filter down and focus on application-specific logging. Trace correlation is an important part of this, to get and end-to-end overview of operations.

To view logs, connect your App Insights instance to a Log Analytics workspace. Within the workspace, General > Logs will provide access to the query editor — you can either user one of the default Queries pop-up or write your own.

For example, to see all recent traces, and the correlation between them you can use a query like:

union AppTraces, AppDependencies, AppRequests
| where TimeGenerated > ago(30m)
   and Properties.CategoryName !startswith "Microsoft"
| sort by TimeGenerated desc
| project TimeGenerated, Type, OperationId, Id, Properties.SpanId, 
   ParentId, ClientType, Message, Name, SeverityLevel, Properties, 
   Properties.CategoryName, OperationName, SessionId,
   UserId, AppRoleInstance

Example output:

This example shows all the traces from one operation are linked to the same OperationId 029c3..., and the parent-child relationship between two tiers client (Browser) and server (PC) can also be determined:

  1. Client (Browser) AppTraces have a ParentId 7d65e...
  2. The client has a link from this parent to a child AppDependency with Id 73676...
  3. On the server (PC) the dependency is recorded as the parent if the AppRequest Id 15c7e...
  4. Additional traces on the server show the request as the ParentId (and there may be further parent-child links depending on the number of tiers).

There are many other types of records that can be queried, for example developers may often be interested in exceptions and traces that feature a particular keyword:

union AppExceptions, AppTraces
| where TimeGenerated > ago(30m)
| sort by TimeGenerated desc
| search "Password"
Continue reading App Insights trace correlation