blog

What can cause the trigger for a Power Automate Flow to stop firing?

Tue, 06 May 2025 09:58:02 GMT

When the AAD account that owns a Power Automate flow (or Cloudflow, or whatever the name is by the time you read this....) is disabled it sometimes causes the trigger for the flow to stop working, even if you've changed the owner of the flow. Helpful, I know.

It looks like this is caused by the trigger being still owned by the disabled user, which you can check by connecting via SQL to your Dataverse environment and running this query:

select * from callbackregistration where ownerid ='THE_ID_OF_THE_DISABLED_USER_HERE'

This will return a list of all the callbackregistration's (aka: Flow triggers) that are still owned by the disabled user. The fix that we found for this is to, for each flow:

Stop the flow - it has to be stopped to be able to delete the trigger
Delete the trigger
Restart the flow

The trigger will get re-created when the flow is restarted.

The easiest way to delete the trigger is to create a flow and plug each id in that you need to clear, like this:

"There is no Functions runtime available that matches the version specified in the project." when moving an Azure Functions project to a different machine

Tue, 14 Jan 2025 15:20:45 GMT

If Visual Studio gives you the error "There is no Functions runtime available that matches the version specified in the project." then you'd, reasonably, think that the answer would be to either check for an updated version of Visual Studio or install an additional component. Neither of these are the solution.

In this instance the Azure Functions project has a target framework of net9.0 which is the latest and greatest version. After checking the Visual Studio Installer for updates, and comparing what felt like the right components against the machine that the project came from I decided to try creating a new project and then diff them to find out what the issue was. This is what I saw:

No option for .NET 9.0...... I double checked on the original machine that the project was created on and the drop-down does contain ".NET 9.0 Isolated". This jogged my memory and the answer is.....

Open Visual Studio
Go to Tools > Options
In the Options window navigate to Project and Solutions > Azure Functions
Click on the 'Check for updates' button:
Wait a few moments and a(nother) button should appear below it:
Click on 'Download & Install' and wait

The button should disappear, and the other two (original) buttons will be disabled for a time whilst Visual Studio downloads updates in the background. When the updates have finished downloading the buttons will be re-enabled:

You should be able to now both run your project and create new projects that are targeting the .NET 9.0 (or whatever version you're trying to use where you've seen this error!).

"Changing" the secret for a Dataverse connection in Power Apps

Sun, 05 May 2024 21:37:00 GMT

For reasons best known to Microsoft, you can't change the secret used to authenticate a Microsoft Dataverse connection using a service principal (If you're not familiar with setting these up there's a good guide by Matthew Devaney). That's a bit of a pain when the secret expires, to say the least. There are some workaround for this, like having a very long lived secret, but that doesn't resolve the underlying issue. There is a 'Switch account' button that makes you think "oh, I'll click that and be able to re-authenticate", go on, try it....

Clicking on the button makes a pop-up window very briefly appear, and not a lot else.

It turns out that there is a way to deal with this, albeit it's a bit of a hack and it does rely on you using connection references in your flows. The process consists of:

Creating a new secret
Creating a new service principal backed connection
Updating connection reference(s) that use the old connection
Deleting the old connection
Deleting the old secret

I'm going to assume that steps 1, 4, and 5 are simple enough that there's no need to go into detail. Instead, I'll focus on steps 2 and 3.

When you create the new secret, give it a name that'll let you tie it to the Dataverse connection you're going to create later. If the app registration is called "Dataverse Automation", you could call it "Dataverse Automation (May 2024)" to uniquely identify it.

Creating a new service principal backed connection

This is where the hack, workaround, bodge, call-it-what-you-will, comes into play. There genuinely doesn't appear to be any way to change the secret associated with a service principal backed connection, so the only thing for it is to create a new one. Now, what you don't want to do is click on the "+ New connection" button when you're viewing the list of connection:

There's no option there to create a service principal backed connection. Thanks, Microsoft!

Instead, you want to start creating a new flow (stick with it!):

In the "Build an automated cloud flow" window, give the flow any name you want as it's not going to stick around. Make sure you choose a Microsoft Dataverse trigger, I've opted for "When a row is added, modified or deleted":

Once the flow designer has finished loading you should be presented with the basic starting point of a flow that has nothing other than a trigger. This is the point at which we'll be able to create a new connection, using the new secret, to refresh the connection reference. To do this, click on the "..." menu at the top-right of the trigger and choose "+ Add new connection" and you'll see this:

From here, click on "Connect with service principal" at the bottom of the box and you'll be prompted to give the connection a name, enter the Client ID, Client Secret and Tenant for the connection. Give the connection a name, which is where the name you gave the secret comes in handy. In this example I'm going to call the connection "Dataverse Automation (May 2025)":

Click on 'Create' and, all being well, you'll be returned back to the previous step where you've got an empty flow with a trigger ready to be setup. Close the flow as you don't need to save it. Now that you've got the new connection, with the updated secret, it's time to update the connection reference(s) that use it.

Updating connection reference(s) that use the old connection

Now you've got a new connection for your service principal, go into your solution, into "Connection references" then click on the one that you want to update, so you see this:

Change the selected connection to the newly created one (in my example I'm changing from "Dataverse Automation (January 2025)" to "Dataverse Automation (May 2025)") and then click Save. You'll be prompted with a "Save changes to this connection reference?" message, click the "Save changes" button to continue.

After a few minutes (which the confirmation prompt mentions), everything will have been updated. You can load one of your flows and take a look at the "Connction References" box at the top-right to confirm:

The only thing left to do now is to delete the old connection (I said I wasn't going to mention this, but as there's a useful thing to mention, I'm going to), at which point you might see the Power Apps UI claim that it's still in use:

This is a fib, which is particularly obvious here to me as I deleted "My basic flow" ten minutes before taking the screenshot above...

:::: some time passes ::::

Even an hour later the Power Apps UI is still claming that the old connection is in use in both the flow that still exists (which is using the new connection via a connection reference) and the one that was deleted quite some time ago. It is safe to delete the connection, and that's exactly what I've done

In conclusion

Yes it's a work-around, but in the absence of a way to update an existing connection it's about as good as it gets. It's much better, at least in my opinion, than using a long lived secret. It also reinforces that having everything contained in a solution is the approach to take, as you can't use connection references outside of their solution. If Google, Bing, or your search engine of choice, has landed you here then hopefully this has helped!

Backing up backups from the HRMC 'Basic PAYE Tools' using NTFS junctions and OneDrive

Mon, 29 Jan 2024 17:20:17 GMT

I've been using the HMRC provided 'Basic PAYE Tools' to run payroll and it does the job adequately. The one gap is that, whilst it performs periodic backups, backups are to a location that cannot be changed. Ideally it would be possible to configure the backup location so it could be somewhere that One Drive covers automatically, but at the time of writing the configuration is static.

The quickest and easiest way to achieve this, as OneDrive doesn't support backing up entirely arbitrary locations, is to use the mklink tool to create an NTFS Junction between the folder that contains the Basic PAYE Tools backups and a folder inside OneDrive. The command to run is:

mklink /j "C:\users\robertwray\OneDrive\RTI Backup" "C:\Users\robertwray\AppData\Roaming\HMRC\payetools-rti\backups"

This creates a new folder called RTI Backup inside my OneDrive folder (this works equally well with "personal" and "business" OneDrive) which is a mirror of the folder where the Basic PAYE Tools backups are found.

Remember: Deleting a file in one folder will delete it in the other, it is not a copy of the backup folder, they're two views of the same files/folder.

Within a few minutes copies of all the backup files should be copied to OneDrive and I'm sure this would work equally well for any other application that's very prescriptive about where it stores its data.

Querying a SharePoint list via Microsoft.Graph after upgrading to .NET SDK v5

Wed, 27 Dec 2023 22:59:17 GMT

I've just finished upgrading all the project references in a CRM solution I built a while ago, which was using the Microsoft.Graph v4 .NET SDK (via the Microsoft.Graph v4.35.0 NuGet package, to be precise), which meant I took an upgrade to the latest version (v5.37.0 at the time of writing) which comes with some breaking changes. The changelog and upgrade guide is a good starting point but the one bit that took a little bit of jiggling to get working again was retrieving items from a SharePoint list.

The original code looked like this:

public async Task<IEnumerable<Order>> GetOrders()
{
    var client = await GetGraphApiClient();

    var queryOptions = new List<QueryOption>()
    {
        new QueryOption("expand", "fields(select=*)")
    };

    var list = await client.Sites[SiteId]
        .Lists[OrdersListId]
        .Items
        .Request(queryOptions)
        .GetAsync();

    var items = from item in list
                select GetFromListItemAdditionalData(item, item.Fields.AdditionalData);

    return items.ToList();
}

public Order GetFromListItemAdditionalData(ListItem item, IDictionary<string, object> additionalData)
{
    var title = additionalData["Title"].AsJsonElement().GetString();

The revised code is:

public async Task<IEnumerable<Order>> GetOrders()
{
    var client = await GetGraphApiClient();

    var list = await client.Sites[SiteId]
        .Lists[OrdersListId]
        .Items
        .GetAsync(requestConfiguration =>
        {
            requestConfiguration.QueryParameters.Expand = ["fields($select=*)"];
        });

    var items = from item in list.Value
                select GetFromListItemAdditionalData(item, item.Fields.AdditionalData);

    return items.ToList();
}

public Order GetFromListItemAdditionalData(ListItem item, IDictionary<string, object> additionalData)
{
    var title = additionalData["Title"] as string;
    ..
    ..
    return new Order(title, ... ...);
}

The bit that took some digging to find the requried tweak was the addition of a "$" prefixing "select" in the expand query definition. The other changes were:

Changing from using QueryOption to QueryParameters.Expand to pull all the additional list columns (i.e. the user columns in the list like title, order id, order date, etc,..) in the results returned
Having the LINQ query operate on list.Value instead of list to process all the list items
No longer needing the call to AsJsonElement to operate on the "AdditionalData" elements

There's nothing particularly complicated, or ground-breaking, there... it did take more searches and digging around than i would like to come up with code that both compiled and worked after upgrading to v5 of the Microsoft.Graph .NET SDK though! Hopefully the note about changing "fields(select=*)" to "fields($select=*)" will help someone out!

Feature flags in ASP.NET Core

Sat, 23 Apr 2022 15:07:05 GMT

I've been working on a few ASP.NET Core applications recently which has given me cause and opportunity to gate some functionality access behind feature flags whilst it's a work in progress, rather than having it gated behind if (app.Environment.IsDevelopment()) or similar where I then have to make code changes to enable the functionality. I've been doing this by using the Microsoft.FeatureManagement.AspNetCore NuGet package which serves me with absolutely everything I need, at least for my relatively simple use cases. The readme on GitHub gives a pretty complete summary of how to use the package but for the sake of completness, here's a basic example excerpted from the implementation I'm using right now.

Adding support to an ASP.NET Core 6 project is as simple as adding the library and then calling the AddFeatureManagement extension method:

using Microsoft.FeatureManagement;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddFeatureManagement();

With that complete it's almost trivial to gate functionality in markup in a razor view behind a flag:

<feature name="SearchInTopHat">
    <form class="d-flex float-md-end" asp-controller="home" asp-action="search">
        <input class="form-control me-2" type="search" placeholder="Search" aria-label="Search" id="q" name="q">
        <button class="btn btn-outline-light" type="submit">Search>/button>
    </form>
</feature>

I've used strings for feature name (the examples in the documentation use an enum which does make for stronger typing in your code but doesn't fully eliminate the risk of typos as it doesn't flow through to the configuration file. In the configuration file the absence of a feature flag implies that it's false which I use to leave all new/un-finished features switched off in production but then enable them in development by listing them in configuration explicitly:

  "FeatureManagement": {
    "SearchInTopHat": true
  }

Documentation for the tag helper can be found at docs.microsoft.com (code: on GitHub). Finding the documentation is mildly tricky as the it's structured by namespace whereas you'll never actually reference the namespace, for the most part - the @addTagHelper directive that needs to be added to _ViewImports.cshtml is usually of the just name the assembly and pull in everything variety. However, once you've found the documentation you can see the different ways the tag helper can be wrangled. Of particular use is the 'Negate' property which allows you to show something when a feature is disabled.

I've only made use of it in a .NET Core web application but it's equally usable in other .NET Core scenarios and has a lot more to it than the surface-level functionality I've described - but that's probably what 80% of people, like me, looking for this will need. It's definitely better than baking your own feature flag functionality and far, far, quicker!

Where have the last two years gone!?

Sun, 17 Apr 2022 10:28:00 GMT

Well... Somehow it's ended up being two years since I last posted (and that was a brief one about resetting your password via the command line) on the blog here - hopefully with the amount of stuff that's been going on from a techy perspective I'll be able to put pay to that for the forseeable future as there's plenty I have to write about in the .NET (Core), Dynamics 365 CRM and Azure space from both a work and non-work perspective.

Other than that, over the last two years I've:

Lost 2¾ stone (or just under 18kg in new money) in weight, primarily over the first 6 to 9 months of the pandemic - more through calorie deficit from food than from exercise
Got a lot better at reducing food waste - preparing three meals a day at home instead of 1 on most days has helped here, I think
Started running

It's the running that's the oddest one of the lot; I've always gone for a walk, however brief, at lunchtime to help me "reset" ready for the afternoon so I'm not someone tho whom exercise is an alien concept. I also spent a couple of years cycling to/from the train station before I finally bit the bullet and passed the driving test.

The first run I did was on June 15th 2020 where I managed ⅓ of a mile before deciding enough was enough and very nearly throwing up. Between then and now I've improved my pacing from about 10:20/mile to about 06:58/mile which I can sustain for a 10KM distance - I've even managed to push that as low as 06:28/mile over a 10KM distance. The biggest running achievement was probably managing a half-marathon (of distance, this was a solo endeavour) in late January this year in 01:38:14 even if my feet did hate me afterwards!

So lots of non-techy stuff which makes a refreshing change and is probably what's helped the last couple of years not seem quite as much of a shocker as they could've otherwise been.

Resetting your password via the command line

Fri, 17 Apr 2020 05:59:00 GMT

If, like me, you're working remotely at the moment you might be spending a non-trivial amount of time using Remote Desktop to hop on and off various machines. It might be that, also like me, you have to access machines that are in a DMZ using machine specific logins. If that's the case you can't just hit CTRL-ALT-END to change your password for the DMZ machine when it's multiple levels of remote desktop down (or if you can, lucky you - it doesn't work for me!). The tool I've been using is the NET command. The syntax for this command to change your password is:

NET USER username *

Substitute your username for username in the command and enter your new password when prompted (twice), and that's that done.

You could also open Control Panel > Administrative Tools > Computer Management and navigate to the System Tools > Local Users and Groups > Users node to change your password but that would need the server to have the Desktop Experience available (i.e. not a 'Core' installation).

Why my Pi-hole was logging tens of thousands of queries against my default gateway for domain SRV records

Sun, 22 Mar 2020 06:00:00 GMT

If you're running Pi-hole (which you should if you can, it's awesome - you can even run it on a virtual machine) and also happen to be running it in an Active Directory environment, there are various recommendations regarding how to configure it dotted around on the internet.

Even having followed everything I've read, I was seeing some very unusual traffic where there would be regular incredibly large batches of SRV queries against some of the domain DNS entries like _ldap._tcp.dc._msdcs.mydomain.local which looked more than a little odd. These queries would tick-up at a rate of knots, 10s of them a second, and all were showing as originating from my UniFi USG device despite the fact that this would have no cause to be making these requests. Because I run a lab environment with multiple domains, subnets and VLANs my immediate suspicion was that one of the secondary domains had got itself a little broken but the inter-domain trust appeared fine and there was a distinct lack of errors or warnings in the Event Log in any domains that may be involved.

Eventually (by trial and error, sitting, watching and waiting) I narrowed it down to having the Use Conditional Forwarding setting in the Pi-hole configured to query the USG:

Changing this to the IP address of one of the primary domains controllers stopped this massive number of queries from being logged with no negative impact on domain behaviour. This also substantially reduced the rate at which logs on the Pi-hole grew, for obvious reasons.

SQL Server Reporting Services subscriptions fail to send email

Wed, 18 Mar 2020 06:00:00 GMT

We've recently had reports that some scheduled reports that are emailed out haven't been sent/received, looking at SSRS the error that was showing was:

Failure sending mail: The report server has encountered a configuration error. Mail will not be resent.

It turns out that the active directory account for the owner of the subscription was recently deleted. Re-creating the subscriptions with the owning account setup as an active user resolved the issue, though longer term we're going to move to getting users to manage their own subscriptions so that we don't have to keep track of, and refresh, subscriptions when the person who created them (invariably someone within IT) leaves the business.

Recovering access to a SQL Server that you don't have 'sa' access to anymore

Sat, 01 Feb 2020 23:58:00 GMT

It's happened to most all of us at one time or another; an instance of SQL Server has been inherited where no one knows the sa password, or is a member of the sysadmin role, it's annoying when it's a SQL Server instance on a Development or Test server, but becomes far more problematic when it's on a production system. Fortunately, there's a way to solve this.

Generating the bootstrap carousel on my homepage

Tue, 21 Jan 2020 07:12:00 GMT

Back in 2017 I wrote about bootstrapping a carousel onto my homepage (was it really that long ago!?) and ever since then I've been adding, removing and modifying items in the carousel by copying, pasting and editing chunks of HTML. Needless to say this is something that's been somewhat error prone, brittle and has made me quite reluctant to modify the carousel unless I absolutely had to. A little while ago I saw a post on meta.stackexchange.com that has a listing of all the data-dumps that are prepared from the Stack Exchange network, but what does this have to do with carousels? Well.... the list is generated from a JavaScript script which avoids the need to hand-crank the markup whenever a new data-dump is generated. This was a "light-bulb" moment, which is a bit embarassing for someone who's generally a massive "automate when you can" kinda person.

To do this I've created a simple JavaScript file in Visual Studio Code to which I do the following to generate the markup:

Open it
Hit F5 to start debugging
Choose 'Node.js' when asked which environment I want to run the script in (I'm not sure if this is something that's present in VS Code by default or something that I've added at some point!)
Copy the markup out of the Debug Console

It's quick and dirty, but it's a lot less painful than editing the markup by hand.

Describing the carousel entries

First up is the script that I use to describe the carousel entries. Each carousel entry consists of an image, a title and a description:

class CarouselItem
{
    constructor(imageSrc, postSrc, title, description, inverse)
    {
        this.imageSrc = imageSrc;
        this.postSrc = postSrc;
        this.title = title;
        this.description = description;
        this.inverse = inverse === undefined ? false : true;
    }
}

They also contain an inverse flag which I use to flip the colour of the text because some of the images lend themselves to having white text instead of black, for readability. There's not a lot else to say about this class structure, so next up is an example of a couple of populated items:

const carouselEntries = 
[
    new CarouselItem('/Media/Default/CarouselImages/bolognese_pasta_bake.jpg',
    '/recipes/lamb-hotpot', 'Bolognese Conchigli pasta bake',
    'The conchigli that I use for this pasta bake are quite thick, lending them an almost meaty texture that really gives the pasta bake some substance'),

    new CarouselItem('/Media/Default/CarouselImages/sea_bass_parcel_carousel_2.jpg',
    '/recipes/sea-bass-parcels-with-asian-flavours', 'Sea Bass parcels with asian flavours',
    'Sea Bass parcels with garlic, ginger, mirin and other goodies')
]

For the purposes of this post I've not listed each and every entry in the carousel as that would be very long, very boring and very repetitive. The only part of the CarouselItem class that's not used here is the inverse parameter to its constructor, because the majority of carousel items seem to look okay without inverting the colour I've opted to omit this when not required as it makes it more obvious when it is required.

Generating the carousel markup

Because I run this straight in Visual Studio Code and copy the output HTML, the execution harness that comes straight after the carousel entries is pretty simple:

const markup = generateMarkupForCarousel('carousel-example-generic');
console.log(markup);

This runs the generateMarkupForCarousel method, passing in the ID I want to use for the carousel and then sends the generated markup to the console. The code for this method, actually two methods as I have another method I call out to that generates the carousel indicators / slides separately from the container and navigation buttons for the carousel, does a lot of concatenation and looks like this:

function generateMarkupForCarousel(carouselId)
{
    const markupForSlidesAndIndicators = generateMarkupForIndicatorsAndSlides(carouselId);
    const finalCarousel = [];
    finalCarousel.push('<div class="carousel slide" id="' + carouselId + '" data-ride="carousel">');
    finalCarousel.push(markupForSlidesAndIndicators);
    // Controls: Previous
    finalCarousel.push('<a class="left carousel-control" role="button" href="#' + carouselId + '" data-slide="prev">');
    finalCarousel.push('<span class="glyphicon glyphicon-chevron-left" aria-hidden="true"></span>');
    finalCarousel.push('<span class="sr-only">Previous</span>');
    finalCarousel.push('</a>');
    // Controls: Next
    finalCarousel.push('<a class="right carousel-control" role="button" href="#' + carouselId + '" data-slide="next">');
    finalCarousel.push('<span class="glyphicon glyphicon-chevron-right" aria-hidden="true"></span>');
    finalCarousel.push('<span class="sr-only">Next</span>');
    finalCarousel.push('</a>');
    finalCarousel.push('</div>'); // class="carousel slide"

    return finalCarousel.join('\r\n');
}

function generateMarkupForIndicatorsAndSlides(carouselId)
{
    const indicators = [];
    const slides = [];
    // indicators / slides pre-amble
    indicators.push('<ol class="carousel-indicators">');
    slides.push('<div class="carousel-inner" role="listbox">');
    let counter = 0;
    for (const entry of carouselEntries)
    {
        const indicatorCssClass = counter == 0 ? 'class="active" ' : '';
        const slideItemCssClass = counter == 0 ? 'class="item active"' : 'class="item"';
        const titleStyle = entry.inverse ? ' style="color: white;"' : '';
        indicators.push('<li ' + indicatorCssClass + 'data-target="#' + carouselId + '" data-slide-to="' + counter + '"></li>');
        slides.push('<div ' + slideItemCssClass + '>');
        slides.push('<img alt="' + entry.title + '" src="' + entry.imageSrc + '" />');
        slides.push('<div class="carousel-caption">');
        slides.push('<h3><a href="' + entry.postSrc + '">' + entry.title + '</a></h3>');
        slides.push('<div' + titleStyle + '>' + entry.description + '</div>');
        slides.push('</div>'); // class="carousel-caption"
        slides.push('</div>'); // class="item"
        counter++;
    }
    // indicators / slides closing
    indicators.push('</ol>');
    slides.push('</div>');

    const generatedMarkup = indicators.join('\r\n') + '\r\n' + slides.join('\r\n');

    return generatedMarkup;
}

The code could probably be more concise, there's certainly some duplication in the generation of the Previous and Next controls that could be elided and probably a reduction in the amount of code overall.

What's next?

As this is "just" a chunk of JavaScript I could probably embed it in the page and have the carousel generated when the page loads. This would certainly help with page load time, as it looks a little like this at the moment:

Whilst all the images are being loaded in parallel, loading is being triggered before the rest of the page assets, including jQuery which sits a little bit further down. I should probably also look at optimising the images as that's 5.1MB of images being downloaded there which is a little sub-optimal.

Reactive Extensions - Their NuGet packages were renamed

Mon, 20 Jan 2020 08:24:12 GMT

This is an aide-memoire which may be of use to others, though it's old news and almost certainly won't be relevant unless you've got some relatively old codebases that need to be maintained and supported..

According to a post on the RavenDB google group, Reactive extensions announced that:

The NuGet packages have changed their package naming in the move from v2.x.x to v3.0.0

~~Rx-Main~~ is now System.Reactive

~~Rx-Core~~ is now System.Reactive.Core

~~Rx-Interfaces~~ is now System.Reactive.Interfaces

~~Rx-Linq~~ is now System.Reactive.Linq

~~Rx-PlatformServices~~ is now System.Reactive.PlatformServices

~~Rx-Testing~~ is now Microsoft.Reactive.Testing

This brings the NuGet package naming in line with NuGet guidelines and also the dominant namespace in each package. The strong name key has also changed, which is considered a breaking change. However, there are no expected API changes, therefore, once you make the NuGet change, no code changes should be necessary.

Also mentioned in the announcement is the fact that the old packages have been unlisted, meaning that you can find them directly if you construct an appropriate URL containing their old name but a search won't return any results.

Converting a load of files from UTF-16 to UTF-8

Thu, 05 Dec 2019 09:44:31 GMT

There's a database here that I've been meaning to get into source control for a long time. Luckily we have regular backups of it (that have been restore proven) and it never changes so it's been pretty low down the priority list. Along comes today and I need to make some changes so, time to get it all scripted. I do this using SQL Server Management Studio, scripting one object per file and dropping each object type into separate folders. I usually remember to script it out as 'ANSI Text' but this time round I forgot, hence GitHub Desktop showing me this dreaded message.

For various reasons scripting this database is a convoluted exercise, meaning that this one small mistake meant it would take me the best part of an hour to do again. Enter C# and a teeny tiny .NET Core console app (though equally valid as a .NET Framework app as it's not using anything .NET Core specific) to re-save all the files. Without further ado, should the code be useful to you, here it is:

using System;
using System.IO;
using System.Text;

namespace ConvertToUTF8
{
    class Program
    {
        static void Main(string[] args)
        {
            var folder = args[0];

            var files = Directory.GetFiles(folder, "*.sql", SearchOption.AllDirectories);

            foreach (var file in files)
            {
                Console.WriteLine($"Processing {file}");
                var content = File.ReadAllText(file);

                File.WriteAllText(file, content, Encoding.UTF8);
            }
        }
    }
}

This iterates over every .sql file in the folder passed in as the first argument to the app, and its sub-folders, and re-saves the file as UTF-8 instead of UTF-16. Five minutes to write the app, 10 seconds for it to run and that means the best part of an hour saved. Awesome!

Changing a Git branch that's tracking a remote

Mon, 18 Nov 2019 09:59:45 GMT

Having done some work against a local branch that's associated with a remote for a certain piece of work I've decided I want to keep the work (it's given me some proof-of-concept ideas) but start again on the actual task at hand. Keeping the current branch with all my commits but being able to "reset" back to the branch as it is on the server is the ideal outcome, which I've noted how to do below.

As with almost anything else Git related, there's a lot of places where this answer already exists I'm sure, but for my own reference (and perhaps as someone else might find this as a result of searching and find use for it) here are the steps to take a local branch that's tracking a remote one, rename it and stop it tracking the remote.

With the branch in question being current (which will show in green with an asterisk next to it when running git branch from the console), run the following commands:

git branch -m new_name_for_branch
git branch --unset-upstream

The first command renames the branch and the second command, well, unsets the upstream branch associated with the local branch.

The anatomy of a FIX message

Thu, 31 Oct 2019 15:41:20 GMT

In my current role I've spend some time working with the FIX protocol, which is:

an electronic communications protocol initiated in 1992 for international real-time exchange of information related to securities transactions and markets.

I'm pretty lucky in that there's an awesome library called QuickFix/n that takes care of doing the grunt work for me, but it's never a bad idea to have some understanding of what's happening "on the wire". To that end I took time to pick apart a FIX message using this awesome repository of information at onixs.biz to do so.

The message in question was a request for a quote for a foreign exchange transaction for the EUR/GBP pair, which looks like this:

8=FIX.4.29=17135=R34=325749=COMP-PRICES52=20180508-09:02:43.96856=BANK-PRICES
131=Q-EURGBP-BUY-3357-636613669639680362146=155=EUR/GBP15=EUR38=335740=C
54=164=20180508167=FOR10=150

The message is constructed as follows:

Field	Value	What it is
8 “BeginString”	FIX.4.2	Identifies the start of a message and specifies the protocol version being used
9 “BodyLength”	171	The length (bytes) of everything up until the field “CheckSum”
35 “MsgType”	R	The type of message: Quote Request
34 “MsgSeqNum”	3257	The message sequence number. This increments as each message goes back and forth
49 “SenderCompId”	COMP-PRICES	The identity of the sender of the message
52 “SendingTime”	20180508-09:02:43.968	The time we sent the message
56 “TargetCompId”	BANK-PRICES	The identity of the recipient of the message
131 “QuoteReqID”	Q-EURGBP-BUY-3357-636613669639680362	Unique identifier for the quote
146 “NoRelatedSym”	1	The number of repeating symbols specified
55 “Symbol”	EUR/GBP	The currency pair we’re asking for a quote in
15 “Currency”	EUR	The currency used for price
38 “OrderQty”	3357	The quantity to be used (the amount of money)
40 “OrdType”	C	The type of order. C = “Forex – Market”
54 “Side”	1	2 = “Buy”
64 “FutSettDate”	20180508	The settlement date
167 “SecurityType”	FOR	FOR = “Foreign Exchange Contract”
10 “CheckSum”	150	A calculated three byte checksum based on a calculation referenced in the linked doc

Hopefully this will be useful to someone (other than me)!

Using PowerShell to start and stop Hyper-V virtual machines

Mon, 30 Sep 2019 20:22:50 GMT

I've been doing a bit of VM Host upgrading recently and being able to mainpulate VMs on said hosts via the command line makes things go a lot quicker! Here's two PowerShell commands I've executed more times than I'd care to mention over the past few days:

Get-VM | Start-VM

Get-VM | Save-VM

The first command iterates over all VMs on the current machine (or another one if you've used Enter-PSSession to remote to another machine) and, well, starts them. The second command iterates over all VMs and saves them to disk - effectively hibernating them so you can bring them back up quickly once whatever VM Host upgrading task you're going through has been completed.

How to force Windows 10 / Server 201x to scan for updates

Wed, 28 Aug 2019 05:00:00 GMT

One of the irritations of Windows 10 and its accompanying server editions is that there's no user interface surfaced to re-scan for available updates. The ability to do this comes in handy if, say, you've forgotten to approve all the relevant updates in WSUS before machines have scanned for them, or, a fresh install of Server 2016 has arbitrarily decided that the most recent cumulative update available is from over a year ago and you want to give it a chance to reconsider its choices. Enter UsoClient ...

From a command prompt (I've habitually run this as Administrator but it may not need that..) runt the following command:

UsoClient StartScan

If you've got the Windows Update user interface open you'll see it starting a fresh scan for updates and any newly applicable updates should pop-up and be available for download and install.

(This post is brought to you by my complete inability to remember this command)

Don't slash your globs the wrong way!

Wed, 24 Jul 2019 06:16:46 GMT

I inherited a couple of AngularJS applications that are built using gulp scripts when I started in my current role. Putting to one side the swathes of repeated code both within the build scripts for the same application (literally huge swathes of code copied and pasted from the script for building the desktop version into the script for building the mobile version) and that had subsequently been copied to other applications, these scripts were, a bit shit. To put it diplomatically. Over the past month we've factored up and out code that's common to building all applications and overall reduced the complexity of the build process - who wouldn't want a simpler build process that produces exactly the same end result, right?

The biggest problem we've had is that all projects were using what can only be described as a franken-version of gulp. Version 4.0.0-alpha.2, to be precise. Upgrading from this version to version 4.0.2 resulted in some very weird things happening around file copies; files that started in app\images\ and should end up in .dev\images\ were ending up in .dev\images\app\images - not helpful! Here's a snippet of the build script in question:

pump([
    gulp.src(['app\\images\\**\\*.*']),
    gulp.dest('.dev\\images')
] , done);

I asked a question on Stack Overflow back at the beginning of June, and after offering a bounty a few days ago finally got an answer... When building gobs, use / as a delimiter, not \ or bad things will happen. Changing the direction of all the slashes resulted in a build that gave the same output, but wasn't using a locally cached (the alpha version doesn't exist on npm!) copy of the package, in a simplified test-harness.

The last hurdle to jump was the fact that these build scripts are run on Wndows and the majority of the globs are generated by using path.join to concatenate their constituent parts. This always gives results that contain \ which complicates things, especially as the scripts contain hundreds of instances of this. Our solution, for now, was to replace:

const join = require('path').join;

With:

const path = require('path');
const join = (...args) => path.join(...args).replace(/\\/g, '/');

With that in place, the main build scripts are now using a "real" version of gulp, the builds are doing what they're expected to and all is once again well. For what it's worth, having dug into the changes it looks like the use of a later version of glob-parent by gulp 4.0.2 is responsible for the change in behaviour.

TL;DR: Use / in globs, not \

Refreshing the Windows icon cache

Thu, 13 Jun 2019 07:52:00 GMT

I just updated to the latest and greatest version of Visual Studio Code, but alas my taskbar was still showing the old icon:

From this GitHub comment, here's a quick way to force the shell to refresh its icon cache, [Win]-[R] > ie4uinit.exe -show and there it is: