Very Basic Introduction To The KNMI Climate Explorer

I’ve moved to WordPress.  This post can now be found at Very Basic Introduction To The KNMI Climate Explorer

########################
UPDATE (January 2, 2010): I just received a reply to an email, and the always-helpful Geert Jan van Oldenborgh of KNMI suggested that I replot Figure 15 using the “gridbox” option instead of “shaded”. The “gridbox” option provides a better indication of the data location. The “shaded” option interpolates, it does not extrapolate, “so isolated (rows) of values are not drawn.”

UPDATE 2 (January 5, 2010): Fixed the link to the Monthly observations webpage at KNMI Climate Explorer.

XXXXXXXXXXXXXX

Dr. Geert Jan van Oldenborgh of the Royal Netherlands Meteorological Institute (KNMI) created and maintains (in addition to all of his research endeavors) the web application called the KNMI Climate Explorer. It allows users to perform statistical analysis of climate data. There are a multitude of datasets and analyses available, and any attempt on my part to describe what is available would not do justice to the efforts that have gone into the tool. Many of the datasets are updated monthly. Some are not. And for some datasets, the source may not update the data for a month or two. HADISST always lags the other SST datasets by a month. You can even investigate some of the climate model outputs used for the IPCC AR4.

Readers here know that I use the KNMI Climate Explorer to prepare the vast majority of my posts and my comments in blogs. I use it for data, correlation maps, anomaly maps for comparisons and animations, etc.

Begin with the Climate Explorer: Starting point and explore. Refer also to the note and link on that page that reads, “Some restrictions are in force, notably the possibility to define your own indices, to upload data into the Climate Explorer and to handle large datasets. If you want to use these features please log in or register.”

VERY BASIC INTRODUCTION
So you want to see, for example, if there is any evidence of the 1976/77 Pacific Climate Shift in tropical Pacific Sea Surface Temperature (SST) anomaly data. The KNMI Climate Explorer is a coordinate-based system, so you’ll need to know the coordinates of that area. Based on a map (Figure 1) you elect to use 20S-20N for the latitudes and 120E-90W for the longitudes.


http://i53.tinypic.com/20zsndd.jpg
Figure 1

Open the Monthly observations webpage (Figure 2). There are combined land plus sea surface temperature datasets at the top. Scrolling down, there are datasets for many other variables: land surface temperature, sea surface temperature, marine air temperature, lower troposphere temperature, precipitation, etc. The “more information” buttons (i) to the right of each are links to the web pages of the sources.
http://i53.tinypic.com/245ftih.jpg
Figure 2

Looking at the available SST datasets, Figure 3, you decide to use HADISST. It’s a good choice for a long-term SST dataset, because it is spatially complete (they infilled all of the missing data) and the raw data is reinserted after the infilling. Click on HADISST, and hit enter.
http://i55.tinypic.com/2ezl4p4.jpg
Figure 3

That brings you to the “Field” page, Figure 4. Enter the coordinates you’ve selected: 20S-20N, 120E-90W. There are different ways they could be entered. Some will give you the correct results. Others will not. And there are other ways to investigate the dataset, so it is best to standardize on a method that will work elsewhere.

For example, there is a map-making tool “Plot this field” under the right-hand menu heading of “Investigate this field”. There, Climate Explorer requires you to enter the southern latitude of the area you’re investigating in the left-hand field and the northern latitude in the right-hand one. If you don’t, it won’t produce a map. Latitudes south of the equator are input as negative numbers. That is, if you were looking for the SST data for the latitudes of 70S-40S, you’d enter -70 in the left-hand field and -40 in the right. Likewise, west longitudes are input as negative numbers. That is, 70W is -70.

The western longitude for the area you’re examining is entered in the left-hand field. And the eastern longitude of the area is entered in the right. If you enter them in the reverse order, you’ll get data but it’s not what you want. By inputting the longitudes in reverse order the data would represent the SST of all longitudes except the tropical Pacific.

Crossing the dateline in the map-making webpage also requires that you enter a longitude in the left-hand field that is lower than the right-hand field. That is, you cannot enter 120 (for 120E) in the left-hand field and -90 (for 90W) in the right-hand one, because Climate Explorer will not produce a map. You have to use 120 (for 120E) and 270 (for 90W) or you can enter -240 (for 120E) and -90 (for 90W). If that explanation was confusing, click on “Plot this field” and produce maps for the area outlined in red in Figure 1, using those two examples.

You’ll note in Figure 4 how I’ve entered the coordinates of 20S-20N, 120E-90W. Click on “Make time series”.
http://i53.tinypic.com/343kpeg.jpg
Figure 4

For Figure 5, I’ve changed the zoom on the screen so that all three graphs on the “Time series” page are visible. The top graph presents the data in raw form. For HADISST, the values are the raw SST data. (For datasets such as GISTEMP or CRUTEMP or HADSST2 that are not presented in absolute form, the upper graph will present anomalies). The middle graph is the climatology data used to create the anomalies. And the bottom graph illustrates the anomaly data.
http://i53.tinypic.com/2j4ualy.jpg
Figure 5

Click on the “raw data” link above the top map. The Raw HADISST SST data for the selected coordinates are presented in tabular form, Figure 6. The first column is the year, obviously, and the other twelve columns are the monthly SST data, starting with January in the second column.
http://i54.tinypic.com/if94r8.jpg
Figure 6

Go back to the “Time series” page and click on the “Raw data” link above the bottom (anomaly) graph. The anomaly data is presented in two columns, Figure 7. The first is the month in numerical form and the second is the SST anomaly data for that month. The data is provided as a mix that includes values in scientific notation. (Caution: Do not delete the E-01, or E-02, etc., after the value. Your spreadsheet understands that it’s scientific notation and will accommodate it.)
http://i51.tinypic.com/qn1vls.jpg
Figure 7

Go back to the “Time series” page and scroll down to the “select years” fields under the heading of “Manipulate this times series,” Figure 8. Let’s say you want to examine the data starting in 1950. Fill in both fields and click on “Select”.
http://i52.tinypic.com/xo0ap0.jpg
Figure 8

The KNMI Climate Explorer default base years for anomalies are the entire term of the data you’ve selected. In this example, they would be 1950 through 2010. I’ve highlighted where that’s shown on the anomaly graph in Figure 9. But let’s say you want to use a 30-year period as the base years--for example 1951-1980 like GISS. Enter them in the “Redisplay the anomalies using the years” fields directly below the anomaly graph, and click “Select.”

http://i54.tinypic.com/op0lqq.jpg
Figure 9

There will only be two graphs on the page: climatology and anomalies. Again, I’ve highlighted the base years in Figure 10.
http://i53.tinypic.com/2ia4ccn.jpg
Figure 10

Click on “Raw data” above the anomaly map and Climate Explorer presents the monthly data, Figure 11. “Select all”, then copy and paste to a spreadsheet. And if you’d like to know how to get the numerical months and monthly data into separate columns in EXCEL, refer to Converting txt Data Into Columns In EXCEL.
http://i52.tinypic.com/33cxces.jpg
Figure 11

A FEW NOTES

Figure 12 shows that there is an extra month added to the end the data. The actual final month for the data in Figure 12 is October 2010, but there is a duplicate of the October value listed in November with the note “# repeat last y to get nice gnuplot plot”. I haven’t asked Geert Jan why the extra month is there, but I’ve assumed that it exists to advise you that the download is complete. Note: Don’t use the data for the extra month in your spreadsheet.
http://i51.tinypic.com/24gogef.jpg
Figure 12

There are datasets that are spatially incomplete. For example, HADSST2 (the other Hadley Centre SST dataset) was corrected for known biases, but it is not infilled. (The Hadley Centre is updating it again and correcting for additional biases. HADSST3 should be available sometime in 2011.) Let’s say you wanted to duplicate the work of Thompson et al when they were investigating the 1945 discontinuity in global HADSST2 SST data. One of the subsets they used was an ENSO index called the CTI (Cold Tongue Index), which represents the SST anomalies of the coordinates 6S-6N, 180-90W. So you select HADSST2, enter those coordinates, then limit the time period to 1940 to 1950. The anomaly curve, Figure 13, shows that there are large gaps in the data for the CTI during that period.
http://i56.tinypic.com/2nvb9s7.jpg
Figure 13

Click on “Raw data” above the graph. There's lots of missing data, Figure 14. Climate Explorer identifies missing data with “# repeat last y to get nice gnuplot plot”. The repeated value is handy if only one month is missing. Then the repeated value can be used to fill in the gap. Or if you like, you can delete the repeated data. But there are gaps much longer than one month in the CTI data during this period.
http://i52.tinypic.com/jau9le.jpg
Figure 14

If you were to make a map of one of the missing months (November 1940), Figure 15, you’d note that there is data, not a lot of it, but there is data in the CTI region.
http://i52.tinypic.com/35jxt8i.jpg

“You can retrieve that data to also reduce the gaps. Back up to the “Field” page, which is where the coordinates are entered, Figure 16. There is a field there identified as “Demand at least: ___ % valid points in this region”. The default is 30%. Click on the “help” (i) button and KNMI explains its use:
"Percentage valid points
“The area average is only considered valid when at least this many valid points are included. Enter a smaller number to get more valid data in the resulting time series, but the quality of these data will be lower. A higher number gives fewer but higher-quality data points.”

So you can reduce the gaps in the data by entering a number in the “Demand at least:” field as low as zero (0), but the quality of the data drops.
http://i51.tinypic.com/e8s3dz.jpg
Figure 16

CLOSING NOTES
I use only a tiny fraction of the capabilities of KNMI Climate Explorer. (I try to write my posts for non-technical people, and sometimes I succeed, so I don’t need all of its tools.) Therefore, I will not be able to answer many of your questions. You’ll find it has capabilities that interest you that I haven’t yet found.

Very Important: Research the dataset you intend to use. For example, a large part of the ISCCP cloud amount data is incomplete over the Indian Ocean before 1998, and, if memory serves me well, that dataset was influenced by volcanic aerosols of the El Chichon and Mount Pinatubo eruptions. If you understand where the pitfalls are in the data, it saves embarrassment after the fact.

Oops, almost forgot. If you missed it, there was evidence of the 1976/77 Pacific Climate shift in the tropical Pacific SST data. Refer again to Figure 9 or 10.

I have found Climate Explorer extremely educational and I thank Geert Jan for it.