PhD Update 20: Like a bad smell.....

Hi again! Another wild blog post appeared. PhD corrections, I have come to realise, have a habit of hanging around long past the time you want to have had them finished and done.

Before we get into all of that though, here's the customary list of posts:

• PhD Update 1: Directions
• PhD Update 2: The experiment, the data, and the supercomputers
• PhD Update 3: Simulating simulations with some success
• PhD Update 4: Ginormous Data
• PhD Update 5: Hyper optimisation and frustration
• PhD Update 6: The road ahead
• PhD Update 7: Just out of reach
• PhD Update 8: Eggs in Baskets
• PhD Update 9: Results?
• PhD Update 10: Sharing with the world
• PhD Update 11: Answers to our questions
• PhD Update 12: Is it enough?
• PhD Update 13: A half complete
• PhD Update 14: An old enemy
• PhD Update 15: Finding what works when
• PhD Update 16: Realising the possibilities of the past
• PhD Update 17: Light at the end of the tunnel
• PhD Update 18: The end and the beginning
• PhD Update 19: The Reckoning

See also Doing a 3-way dataset split in Tensorflow // PhD Aside 3, which I posted since the last one of these PhD update blog posts.

Things have not been easy over the last 9 months, but I am making it through. I can't promise when blog posts will come, but know that I have a lot of ideas and it's just a case of having the energy to write them. See also my fediverse account @sbrl@fediscience.org (rss feed) for smaller updates in between times.

Corrections

One of the things I did not realise when I started my PhD was just how much work you are expected to do outside of your main scholarshiped research period. There's writing the thesis, doing the viva, and, of course, the corrections afterwards.

While I'm not sure how much I can share about the corrections I've been given I can say that the process of completing them has been both long and annoying.

More importantly, it is now coming to a close!

Yep, that's right: I'm almost done with my corrections! I just need a laundry list of people to check them and say they are okay, and then I can finally get this PhD thing over with and move on to cooler researchy things that I'll talk about later in this blog post.

As with my viva, the corrections I received were mainly organisational and big picture stuff in nature, though I also had my fair share of experiment redos to complete (ewwww), which have been very time consuming.

Once everything is done, I do plan on making my thesis available for free here on my website if my institution will let me. I doubt it will get indexed in any scholarly search engines any time soon, but hopefully it will be useful to someone here.

Speaking of, I'd like to share a Cool Graph™ I created whilst doing my corrections:

(Above: A grid of graphs showing the stability of the metrics for my rainfall radar models over 7 runs)

This is, as the caption suggests, a random cross-validation (because anything else would be far too complicated to implement) run of my rainfall radar model I have mentioned before. This was one of the things I was asked to do in my corrections.

The code behind this is kinda cool, as it aggregates a metrics.tsv file from every experiment directory in in a given directory.

As I write this I realise that's kinda confusing, so let me show you what the directory of experiments actually looks like:

+ 202412_crossval-stbl7
    + 2024-12-12_deeplabv3+_rainfall_csgpu_ri_celdice_lr0.00001_us2_t0.1_bs32_crossval-stbl7-A
    + 2024-12-12_deeplabv3+_rainfall_csgpu_ri_celdice_lr0.00001_us2_t0.1_bs32_crossval-stbl7-B
    + 2024-12-12_deeplabv3+_rainfall_csgpu_ri_celdice_lr0.00001_us2_t0.1_bs32_crossval-stbl7-C
    + 2024-12-12_deeplabv3+_rainfall_csgpu_ri_celdice_lr0.00001_us2_t0.1_bs32_crossval-stbl7-D
    + 2024-12-12_deeplabv3+_rainfall_csgpu_ri_celdice_lr0.00001_us2_t0.1_bs32_crossval-stbl7-E
    + 2024-12-12_deeplabv3+_rainfall_csgpu_ri_celdice_lr0.00001_us2_t0.1_bs32_crossval-stbl7-F
    + 2024-12-12_deeplabv3+_rainfall_csgpu_ri_celdice_lr0.00001_us2_t0.1_bs32_crossval-stbl7-G

The experiment series directory (202412_crossval-stbl7) contains 7 different runs, matching the stbl7 part of the experiment series name crossval-stbl7.

The directory names there look pretty complicated, but it's actually just made of up all the parameters that I'm currently interested in for that experiment series. Each part is separated by an underscore _.

So, to break down 2024-12-12_deeplabv3+_rainfall_csgpu_ri_celdice_lr0.00001_us2_t0.1_bs32_crossval-stbl7-C:

• 2024-12-12: The date the (individual) experiment was run. These all ran in parallel on a HPC at my university, hence all the dates are the same even though it takes more than 24 hours to train the rainfall radar model.
• deeplabv3+: The architectural backbone of the model in question - this time DeepLabV3+.
• rainfall: Identifier code for the project, rainfall is the traditional informal and short name I gave to the rainfall radar model.
• csgpu: The place it was trained on. In this case csgpu is a small HPC cluster in my department.
• ri: This marks the start of the experiment hyperparams I'm interested in, in no particular order (but the order usually remains constant for a given project). ri stands for Remove Isolated.
• celdice: The loss function. Cross-Entropy Loss + Dice loss.
• lr0.00001: The Learning Rate, this time 0.00001
• us2: UpScale 2 - a property of the model in which it upscales the input x2 and downscales it just before the output. This improves the fidelity of the output at the cost of a higher memory usage.
• t0.1: Threshold of 0.1 - the delimiter between water and no water. At some point, I want to split into multiple bins.
• bs32: Batch Size of 32.
• crossval-stbl7-C: The experiment series code - see above.
  • crossval-stbl: The main part of the experiment series code
  • 7: The number of cross-validation runs
  • C: The differentiator. In this case it's part C of the experiment series since they are all the same, but usually each model has something unique about it, e.g. regresstest-regress vs regresstest-class.

Hmmm, looking at this it might be a bit more complicated a system than I first expected, but it makes sense to me. I wonder if I've blogged about how I organise experiments already? If not, that should go on the todo list.

Anyway, this is the foundation of my entire organisational system for running experiments. I've developed quite an intricate system since I started running experiments in 2020, but fundamentally it is based on the principle of preserving as much information about any given experiment that I've run as possible, as I am sure to need it later.

Even if I don't think I'll need it!

In fact, especially if I don't think I'll need it, because I've been bitten enough times to know that it's not a case of if, it's most certainly a case of when.

Corrections? What corrections?

Not to get too distracted, while I don't think the University would like it very much if I shared my exact list of corrections, it boiled down to the following basic principles, in no particular order:

• There wasn't a clear narrative carrying the problem forwards through the thesis
• They wanted more experiments running to confirm the stability of the models trained - hence this post on a 3-way split since they wanted a 3-way split, and also hence the stability testing done to produce the graph above amongst others
• They wanted a regression model training for the rainfall radar model and a comparative analysis against my existing classification-based approach

It doesn't sound like much, but it has been quite a lot of work to get to this point, especially since I have been doing more teaching than I expected starting in September last year. I'm glad now that I applied for a 6 month extension and for the help of the people around me (and 2 people in particular - not sure if I can mention your names, but you know who you are), otherwise I would have run out of time to complete my corrections long ago.

Future research

Now that my corrections are (hopefully) coming to and end and I'm starting to get a handle on the teaching I've been asked to do (wow, and that isn't even the half of it), I'm finally starting to get myself into a place in which I can FINALLY start to look forwards to some more research that is actually useful, as opposed to making seemingly endless corrections to my thesis (the social media chapter in particular I can do SO MUCH BETTER).

First of all are real improvements to my rainfall radar model. These improvements largely fall into a few categories:

• Analysing and improving the model's ability to actually predict floods, and applying sample weighting (psst, secret second graph for those of you who are still reading!) to hopefully measuably improve my model's ability to make actually useful predictions
• Swapping out the physics-based model the model is trained on because it's bad and I didn't prepare the data very well and it's all bad
• Exanding the model's ability to predict multiple bins instead of just a binarised water/no-water situation

These are not necessarily in order, but I imagine I'll likely tackle them in something like this order.

On the social media side, I know that I can do so much better than my social media paper which somehow has 41 citations (just HOW??!). Binary sentiment analysis is cute and all, but at the intersection of AI, disaster situational awareness, and user interface (UI) design and user experience (UX) I believe that I can do much better in the organisation of unstructured data.

With the use of contemporary AI algorithms and UI/UX, the extraction and presentation of richer information should be possible.

Even though these research plans won't be part of my PhD, I will still continue blogging about it! Who knows, I might even start a new long-running blog post series to mark the beginning of a new era in my life.

And, of course, I'll continue to share Cool Graphs™!

BlueSky

BlueSky: As a last thing, I'm going to blog about it at some point but I'm now using bridgy fed to allow you to follow me on Bluesky! I'm @sbrl.fediscience.org.ap.brid.gy, and to interact with me you'll need to follow @ap.brid.gy.

BlueSky seems to be becoming very popular, especially my circles - but while it pretends to be decentralised, it isn't. For this reason and others, my primary social media will remain on the Fediverse to ensure and preserve the long-term viability of my account.

I encourage you to join the fediverse too - it's a nice and friendly place :D

Final thoughts

It has been a long road, but I am finally nearing the end of one book and the beginning of another. This is not the last post in this series - I have at least 1 more planned. When I have the energy, I want to talk about my experiences learning to teach (I'm doing a course called PCAP right now, as it was a stipulation of my contract) in what may be a longer blog post than I expect.

I'm looking forward to continuing my research journey and blogging along the way right here at my stardust blog (I think this is the first time I've mentioned my blog's name!).

I'll see you next time, in what might be one of the last blog posts in this series: PhD Update 21: Where the water meets the sky.

--Starbeamrainbowlabs