News

June 22, 2022

There has been much progress made on static mapping in the past month. We've been a lot happier with the results we've been seeing over the past week or so. Taking techniques from the domestic dataset program and applying them to (the imperfect, but very timely) international real-time weather API data streams has yielded satisfactory results without having to resort to increasing overhead by buying data from a quality-checked data provider.

The maps below are certainly cleaner, but I must say that continent-scale resolution leaves me wanting more detail. Most likely, for the final product, the continents will be subdivided into regional maps, which can provide more useful levels of detail (see the map of the southwestern US below).

May 3, 2022

The interactive map is now online for the entire world, as well as the data section. There are a few issues to iron out, mostly involving missing place names and formatting issues. Overall, the system is functioning as intended and it's been interesting to interpret the current heatwave in South Asia through this tool.

As for non-interactive maps, I have succeeded in cleaning up the free datasets and using advanced geostatistical techniques so that mapping is possible. I took a GIS course in the spring of 2020, and needless to say, I didn't quite retain as much information as I could have. After going over some of my old notes, I realized that I did learn how address irregularities in point-based datasets, and all I had to do was learn how to program the same processes in Python that I had been doing in ArcMap.

The results haven't been perfected yet, but they are much better than my previous method and they should be able to be added soon. For example, although there are (at least) two points in the heat advisory plot that seem to be anomalous, they are not warping the maps as they were in my last update.

April 26, 2022

Much progress has been made over the last couple weeks. The last hurdle for me getting the new system online (besides the initial setup) has been data quality. The data source I've been using provides weather data for free but only provides robust quality control for their paying customers. I'll only being needing data on an hourly basis or so, so I may be able to find an alternative source.

Inaccurate data is a problem for mapping because of how the interpolation process works. A good analogy is throwing a rock into a lake. The ripples will extend for some time beyond where the rock was thrown, but other environmental factors will prevent the ripple from continuing indefinitely. If someone is working with a static dataset (for example, of past conditions), there are ways to more easily account for outliers. But this program will be taking in new data on at least an hourly basis. I attempted to program my own quality control procedure, but it hasn't caught everything it needs to.

Below are some examples of data outside the feasible range affecting results. I've highlighted WBGT values above 40 in green because they are outside of a normal, feasible range for WBGT in most places. A malfunctioning weather station in Madagascar is potentially skewing the results for coastal Mozambique as well. In addition, weather data is so sparse in areas (like central Africa), that an error in Sudan is inflating values in the region as well. The second plot was made about five minutes later—the issue in Sudan was caught and resolved, but the issue in Madagascar continued to persist. The issue in Sudan is potentially worse for our purposes; however, because it did not cross the 40°C threshold, making the error less obvious.

I wish I could say that this issue was confined to developing countries, but it's happened off and on in the US and Canada as well, as is shown in the third plot (of the southeastern US) below. The fourth plot shows data from an alternative (free) data source that covers the United States only, which provides much better results. For the rest of world, I will be looking into options for better data.

April 8, 2022

The new backend is almost finished and it's so much faster than the prototype running the site currently. I can process thousands of weather stations' data in a matter of seconds with my new program, which allows me to provide this service to the entire world. During the last week, I've been working on interpolation methods for providing graphics and will move on to getting the web interface sorted out in the coming weeks.

I also sorted out the issue with UTCI values. It won't be fixed in the current version, but the improved method is already written into the new edition.

Here's a preview of the interpolation and graphics I've been refining:

End of March, 2022

The data section now has a search bar so you can search for your city's heat conditions. An error that causes high winds to produce heat advisories will be fixed in the next edition, as well as keeping outdated results from appearing in the map. An error where high winds cause an upward bias in UTCI calculations will be looked into as well. This may be endemic to the polynomial regression method I'm using (see the "wall of math"), or it might be an issue with the way I'm calculating mean radiant temperature.

March 7, 2022

The data section is live! You can now view conditions including air temperature, relative humidity, wet bulb temperature, UTCI, and wind speed. As far as I know, no one else is offering real-time UTCI values elsewhere.

February 18, 2022

Areas of the west coast, Hawaii, and the Great Plains have been added.

The confidence interval issue has been resolved. There was previously an error in the code that skipped the step of calculating the T-score for the distribution.

February 6, 2022

The city of Mérida, Yucatán in Mexico saw an estimated WBGT of 33.1°C and a wet bulb temp. of 31.4°C for a short period earlier this week, which is the first 33+ value I have observed YTD.

This past week saw additions of more Central American weather stations (more of Mexico will be coming soon)