Removing The Effects of Natural Variables - Multiple Linear Regression-Based or “Eyeballed” Scaling Factors

I’ve moved to WordPress.  This post can now be found at Removing The Effects of Natural Variables – Multiple Linear Regression-Based or “Eyeballed” Scaling Factors

#########################
This is the second of a series of follow-up posts to Can Most Of The Rise In The Satellite-Era Surface Temperatures Be Explained Without Anthropogenic Greenhouse Gases?. The first follow-up was Notes On Polar Amplification.

This post discusses the impacts on global temperatures of a natural mode of Sea Surface Temperature variability: the El Niño-Southern Oscillation (ENSO). If this subject is new to you, refer to the post An Introduction To ENSO, AMO, and PDO – Part 1.

INTRODUCTION
My post Can Most Of The Rise In The Satellite-Era Surface Temperatures Be Explained Without Anthropogenic Greenhouse Gases? was cross posted at Watts Up With That? under the same title (Can Most Of The Rise In The Satellite-Era Surface Temperatures Be Explained Without Anthropogenic Greenhouse Gases?) In a step-by-step process, that post illustrated how I was removing the linear effects of El Niño and La Niña events and of volcanic eruptions from the global temperature record during the satellite era. I then went on to explain and provide links to more detailed explanations of the secondary effects of the ENSO process and how they impact the multidecadal trend.

There were a few comments on the WUWT thread about my use of “eyeballed” scaling factors. They wondered what I meant by “eyeballed”, expressed concern about a scaling factor that was not based on statistics, and suggested using EXCEL to determine the scaling factors.

“EYEBALLED” SCALING FACTORS
The scaling and lag I used in comparisons of the global temperatures and the ENSO and Volcano proxies were established by the visual appearance of the two variables, using the larger(est) event (the 1997/98 El Nino, and the 1991 Mount Pinatubo eruption) as reference. “Eyeballed” simply referred to the visual comparison of the impacted variable (global temperature) and the impacting variables (ENSO and Volcanic eruptions). Refer to Figure 1, which is also Figure 1 from Can Most Of The Rise In The Satellite-Era Surface Temperatures Be Explained Without Anthropogenic Greenhouse Gases?. Note how the NINO3.4 data has been scaled (multiplied by a factor of 0.16) and lagged (moved back in time 3 months) so that the rises of the two datasets are about the same during the evolution of the 1997/98 El Niño. Notice also how the response of the global temperature data to the lesser ENSO events after 2000 doesn’t always align with the NINO3.4 SST anomalies.

http://i53.tinypic.com/deuxdz.jpg
Figure 1

The data indicates that the larger events (such as the 1997/98 El Nino) are strong enough to overcome the noise that can mask the global response to lesser events. In other words, I used the larger 1997/98 ENSO event as reference because the response to it was clearest. Statistical methods such as linear (or multiple linear) regression rely on the relationship between two (or more) datasets over the term of the data. Any additional noise in the data during the smaller ENSO events (and during the lesser volcanic eruptions) may bias the results.

If we could determine the cause or causes of that additional noise, then adding those variables to a multiple linear regression analysis would be helpful.

MULTIPLE LINEAR REGRESSION

Analyse-it for Excel is a statistical add-on package for EXCEL. (It has a 30-day free trial period). One of its features is multiple linear regression. Their Multiple linear regression webpage provides a general description of this feature: “Linear regression, or Multiple Linear regression when more than one predictor is used, determines the linear relationship between a response (Y/dependent) variable and one or more predictor (X/independent) variables. The least-squares method is used to minimize the vertical distance between the response and the fitted linear line.”

The response variable discussed in this post is global temperature (represented by GISS LOTI from 60S-60N) and the predictor variables are ENSO (represented by NINO3.4 SST anomalies) and volcanic eruptions (represented by GISS Stratospheric Aerosol Optical Thickness data: ASCII data). Figure 2 shows the differences in the variability of those three datasets.
http://i52.tinypic.com/b7yplu.jpg
Figure 2

According to the EXCEL multiple regression software, the scaling factor to best fit the wiggles of the ENSO data to those in the global temperature data is 0.07262, and for the Aerosol Optical Thickness data it’s -3.191.

Note: The scaling factors determined by the regression software in this post are based on the “raw” data. I’ve used a 13-month running-average filter in the graphs after the fact to reduce the visual effects of seasonal variations and other noise.

In Figure 3, I’ve illustrated the multiple linear regression-estimated relationships between the GISS LOTI data, the NINO3.4 SST anomalies (ENSO), and the GISS Stratospheric Optical Thickness (Volcano) data. The scaling for the Volcano proxy data appears too large, while the scaling for the ENSO proxy appears too small. The global temperature anomaly (60S-60N) response to the 1997/98 El Niño almost doubled the rise in the scaled NINO3.4 SST anomalies.
http://i54.tinypic.com/2zdsltk.jpg
Figure 3

Figure 4 illustrates the result when we subtract scaled ENSO and Volcano proxy data from the Global Temperature data. That dataset is supposed to represent global temperature data that has been adjusted for ENSO and volcanic eruptions. I’ve also included the scaled NINO3.4 SST anomalies to show that much of the ENSO signal remains in the data. Note also how the response in global temperature lags the ENSO proxy.
http://i51.tinypic.com/20ffatc.jpg
Figure 4

Multiple linear regression does not appear to do a good job of removing the impacts of ENSO.

THREE-MONTH LAG

Referring back to Figure 1, we can see that the global temperature response during the 1997/98 El Niño lagged the NINO3.4 SST anomalies by 3 months. Let’s also look at the Volcano proxy. Figure 5 shows it and the ENSO-adjusted GISS Land-Ocean Temperature Index (LOTI) data from Can Most Of The Rise In The Satellite-Era Surface Temperatures Be Explained Without Anthropogenic Greenhouse Gases?. (Figure 5 was Figure 2 in that post.) In order to align the leading edge of the global temperature response to the 1991 eruption of Mount Pinatubo (the larger and clearer response), I had to lag the Volcano proxy 3 months.
http://i51.tinypic.com/8vyd54.jpg
Figure 5

If we shift the NINO3.4 SST anomalies and Aerosol Optical Thickness data three months, the EXCEL Analyse-It software, of course, calculates different scaling factors: 0.08739 for the ENSO proxy and -3.495 for the Volcano proxy. The ENSO- and Volcano-adjusted data using these updated scaling factors (based on a 3-month lag) is shown in Figure 6. Again, much of the ENSO signal remains.
http://i53.tinypic.com/2q1ib7t.jpg
Figure 6

Once again, multiple linear regression appears to have done a poor job of removing the effects of ENSO.

MODEL-PREPARED SCALING FACTORS

In a 2009 paper, Thompson et al created models of global temperature responses to ENSO and Volcanic eruptions. The paper was “Identifying Signatures of Natural Climate Variability in Time Series of Global-Mean Surface Temperature: Methodology and Insights”. Link : http://www.atmos.colostate.edu/ao/ThompsonPapers/ThompsonWallaceJonesKennedy_JClimate2009.pdf

On page 2 they provided an overview of their methods: “The impacts of ENSO and volcanic eruptions on global-mean temperature are estimated using a simple thermodynamic model of the global atmospheric-oceanic mixed layer response to anomalous heating. In the case of ENSO, the heating is assumed to be proportional to the sea surface temperature anomalies over the eastern Pacific; in the case of volcanic eruptions, the heating is assumed to be proportional to the stratospheric aerosol loading.”
The same method was used in its companion paper Fyfe et al (2010), “Comparing Variability and Trends in Observed and Modelled Global-Mean SurfaceTemperature.”
http://www.atmos.colostate.edu/ao/ThompsonPapers/FyfeGillettThompson_GRL2010.pdf

Thompson et al provided a link to their “Global Mean”, “ENSO fit”, and “Volcano fit” data. Link to Data. Figure 7 shows the three Thompson et al datasets. Note: Thompson et al examined the data from 1900 to March 2009. Since we’re only looking at the change in temperature during the satellite era (dictated by the second, more recent of the two SST datasets used by GISS) I had to shift their global mean data down 0.25 degrees C to align it with the other datasets.
http://i53.tinypic.com/286zdpf.jpg
Figure 7

And Figure 8 shows what Thompson et al refer to as the “ENSO/Volcano Residual Global Mean” temperature anomalies after the ENSO and Volcano proxy signals have been removed. And, again, I’ve included the “ENSO fit” data to show how poorly it approximated the “Signatures of Natural Climate Variability in Time Series of Global-Mean Surface Temperature”. The response of their adjusted Global Mean Temperature to the 1997/98 El Niño is greater than their “ESNO fit” data, and this is after the effects of ENSO have supposedly been removed.
http://i51.tinypic.com/2z82jo8.jpg
Figure 8

In summary, linear regression and models prepared for climate studies appear to do a poor job of removing the linear effects of ENSO. If the secondary effects of ENSO were also included in the multiple linear regression, would the results be better?

These secondary effects are easy to see if we look at…

THE RESULTS USING THE EYEBALLED METHOD

As noted earlier, for the “eyeballed” method, I keyed the scaling of the ENSO and Volcano proxies visually off the leading edges of the 1997/98 El Niño and the 1991 eruption of Mount Pinatubo. Refer to Figure 9. In this example, I’ve reduced the scaling on the Volcano proxy data, so that its impact is approximately 0.2 deg C for the Mount Pinatubo eruption.
http://i55.tinypic.com/9iy1xd.jpg
Figure 9

The result when the ENSO and volcano signals are removed is shown in Figure 10. Also shown is the scaled ENSO proxy as a reference. The rises that occur after the 1986/87/88 and the 1997/98 El Niño events make it appear as though there is another lagged ENSO-related signal.
http://i54.tinypic.com/351gxds.jpg
Figure 10

And to see this signal, we invert the scaled NINO3.4 SST anomalies. Refer to Figure 11. That is, we multiply the NINO3.4 SST anomalies by a negative number (-0.1 scaling factor). The inverted ENSO data has not been lagged. But it has been shifted down 0.05 deg C to align it with the 1987/88 upward shift in the ENSO- and Volcano-adjusted global data. The two datasets diverge slightly at times between 1989 and 1996, but the adjusted global temperatures follow the inverted NINO3.4 SST anomalies reasonably well. Then there is a significant divergence during the evolution of the 1997/98 El Niño. The adjusted global temperature anomalies do not drop at that time proportionately to the El Niño. (And there is no reason global temperatures should drop a large amount. The majority of the warm water that fuels an El Niño comes from below the surface of the Pacific Warm Pool.)
http://i56.tinypic.com/2da0dx4.jpg
Figure 11

If we shift the NINO3.4 SST anomalies up 0.21 deg C, Figure 12, we can see that the adjusted global temperatures rise in response to the transition from El Niño to La Niña in 1998 and then, once again, they follow the general variations in the inverted NINO3.4 SST anomalies, but running in and out of synch with it.
http://i54.tinypic.com/msi0br.jpg
Figure 12

In other words, the vast majority of the rise in ENSO- and Volcano-Adjusted GISS Land-Ocean Temperature Index data could be explained by one or more detrended Sea Surface Temperature dataset(s) that mimicked inverted NINO3.4 SST anomalies with upward shifts, similar to what’s illustrated in Figure 13.
http://i54.tinypic.com/19y44y.jpg
Figure 13

I qualified the above statement with “detrended”. At a few alarmist blogs, I received a few negative comments about my Can Most Of The Rise In The Satellite-Era Surface Temperatures Be Explained Without Anthropogenic Greenhouse Gases? post because I had used the trends in the SST subsets to explain the trend in global temperature. I had explained in the post why the upward trends were associated with the process of ENSO, but I received the complaints regardless. Those bloggers, of course, failed to read the explanation (It starts under the heading of “La Niña Events Are Not The Opposite Of El Niño Events”) or had elected to misrepresent my post. But in order to overcome this objection in this post, I’ll use detrended SST anomalies for the following illustrations. The same and other bloggers also complained about the minimal sizes of the Kuroshio-Oyashio Extension and South Pacific Convergence Zone (SPCZ) Extension SST subsets I used in the “Can Most” post. Those areas were used because they had the strongest warming signals during a La Niña. But since the objections exist, I’ll use SST datasets that represent larger portions of the global oceans.

There are two detrended sea surface temperature subsets covering significant portions of the global oceans that have the same upward changes in temperature during those two El Niño to La Niña transitions shown in Figure 13. But if we had relied on the scaling factors suggested by the multiple linear regression, or if we had used a model similar to the one created by Thompson et al, would we have noticed the relationship?

EAST INDIAN-WEST PACIFIC SST ANOMALIES
The first of the SST subsets is the East Indian-West Pacific Ocean. The coordinates of this dataset are 60S-60N, 80E-180E. I’ve highlighted those coordinates in the following map. Figure 14 is a correlation map of annual (January to December) East Indian-West Pacific SST anomalies and annual GISS LOTI data for 1982 to 2010. Much of the Northern Hemisphere land surface temperature anomalies vary with the East Indian-West Pacific SST anomalies. That is, when the East Indian-West Pacific SST anomalies rise in those upward ENSO-induced steps, so do those areas highly correlated with it.
http://i53.tinypic.com/14kz9s4.jpg
Figure 14

I have discussed the East Indian-West Pacific dataset in many posts over the past two years, so I do not intend to repeat the discussion here. Those posts started with Can El Niño Events Explain All of the Global Warming Since 1976? – Part 1 and Can El Niño Events Explain All of the Global Warming Since 1976? – Part 2. The most detailed explanation of why the East Indian-West Pacific SST anomalies shift upwards as a response to those two ENSO events is provided in my series of posts:
More Detail On The Multiyear Aftereffects Of ENSO – Part 1 – El Nino Events Warm The Oceans
And:More Detail On The Multiyear Aftereffects Of ENSO - Part 2 – La Nina Events Recharge The Heat Released By El Nino Events AND...During Major Traditional ENSO Events, Warm Water Is Redistributed Via Ocean Currents.
And:
More Detail On The Multiyear Aftereffects Of ENSO - Part 3 – East Indian & West Pacific Oceans Can Warm In Response To Both El Nino & La Nina Events

And for those who like visual aids, refer to the two videos included in La Niña Is Not The Opposite Of El Niño – The Videos.

Again, to overcome one of the complaints, I need to detrend the East Indian-West Pacific data. Detrending is said to eliminate the “global warming signal”. So I employed the same method used for the Atlantic Multidecadal Oscillation data. HADISST is the long-term Sea Surface Temperature anomaly dataset used by GISS in their LOTI product. The long-term HADISST (1870-2010) East Indian-West Pacific SST anomalies had a linear trend of 0.44 deg C per Century, and that’s slightly higher than the global SST anomaly trend of 0.39 deg C per century. To detrend it, I subtracted the linear trend values for each month from the East Indian-West Pacific SST data.

And as shown in Figure 15, the detrended East Indian-West Pacific SST anomalies could easily explain much of the rise in the ENSO- and Volcano-Adjusted GISS LOTI data since 1982. The similarities between the adjusted global temperature data and the East Indian-West Pacific SST anomaly data are remarkable.
http://i54.tinypic.com/2nalcab.jpg
Figure 15

As discussed and illustrated in the linked posts, the upward shifts in the East Indian-West Pacific SST anomalies are secondary effects of the warm water that was leftover from the 1986/87/88 and 1997/98 El Niño events and the results of the La Niña process itself. Basically, the ENSO processes cause the SST for this part of globe to rise in response to both El Niño and La Niña events. And the effects are cumulative if a La Niña follows an El Niño. The cumulative effect can be seen in the following animation. It shows a series of maps of 12-month average global SST anomalies, and it runs from the start of the 1997/98 El Niño to the end of 1998/99/00/01 La Niña. To its right is a graph of scaled NINO3.4 SST anomalies and the SST anomalies of the East Indian-West Pacific Oceans. The data in the graph have been smoothed with a 12-month running-average filter to match the maps.
http://i51.tinypic.com/2qiwscz.jpg
Animation 1

And for reference, Animation 2 includes the Sea Surface Temperature Anomalies of the rest of the oceans (East Pacific, Atlantic, and West Indian), and this dataset includes the North Atlantic with its Atlantic Multidecadal Oscillation.
http://i51.tinypic.com/b46lph.jpg
Animation 2

The next variable is widely known, but it is often overlooked.

THE ATLANTIC MULTIDECADAL OSCILLATION (AMO)
The second dataset that matches the upward steps in the adjusted GISS LOTI data is the Atlantic Multidecadal Oscillation or AMO data. For those new to the AMO, refer to the post An Introduction To ENSO, AMO, and PDO -- Part 2.

The AMO data used here is detrended North Atlantic SST anomaly data. Again, as noted in the Wikipedia Atlantic Multidecadal Oscillation webpage, “detrending is intended to remove the influence of greenhouse gas-induced global warming from the analysis.” The data is available through the NOAA Earth System Research Laboratory (ESRL) Atlantic Multidecadal Oscillation webpage (the AMO unsmooth, long: Standard PSD Format data).

Figure 16 is a correlation map of annual (January to December) Atlantic Multidecadal Oscillation (AMO) data and annual GISS LOTI data for 1982 to 2010. Like the East Indian-West Pacific SST anomalies, much of the northern hemisphere varies in temperature with the AMO. Again, this means that much of the northern hemisphere surface temperatures rise with the upward steps in the AMO data.
http://i55.tinypic.com/opsbv9.jpg
Figure 16

And for those interested, Figure 17 is a GISS LOTI trend map created at the GISS Global Mapswebpage. It illustrates the rise in surface temperature anomalies from 1982 to 2010. Note the similarities between it and the correlation maps in Figures 14 and 16. For the most part, the regions where the AMO and the East Indian-West Pacific SST anomalies correlate with the Global GISS LOTI data are also where most of the rises in surface temperature occurred. There are a few areas with differences, but the maps agree quite well.
http://i56.tinypic.com/2lar62e.jpg
Figure 17

The AMO data is compared to the adjusted GISS LOTI data in Figure 18. Again, note the agreement between the AMO data and the adjusted GISS LOTI data.
http://i55.tinypic.com/eu0sk0.jpg
Figure 18

And if we compare the ENSO- and Volcano-adjusted GISS LOTI data to both detrended SST-based datasets, Figure 19, we can see that it wouldn’t require a lot of effort to explain most of the global warming from 1982 to 2010 using the AMO and detrended East Indian-West Pacific SST anomaly datasets.
http://i53.tinypic.com/2gt9mic.jpg
Figure 19

Let’s add the AMO and the detrended East Indian-West Pacific SST anomalies to the multiple regression analysis and see how that impacts the results.

MULTIPLE LINEAR REGRESSION WITH ENSO, VOLCANO, AMO, AND EAST INDIAN-WEST PACIFIC SST DATA

The NINO3.4 SST anomalies, the mean optical thickness data, the AMO data, and the detrended East Indian-West Pacific (EIWP) SST anomalies were all entered into the EXCEL multiple linear regression software as predictors with the GISS LOTI data as the response variable. EXCEL determined the scaling factors listed in parentheses in Figure 20. The GISS LOTI data illustrated has been adjusted by those four scaled variables. I’ve also included the scaled NINO3.4 SST anomalies as reference. The low frequency variations in the adjusted GISS data mimic the ENSO proxy before the 1997/98 El Niño. They show little relationship from 1998 to 2007, and then they appear to mimic one another again.
http://i51.tinypic.com/f4g3ns.jpg
Figure 20

Figure 21 compares the unadjusted GISS LOTI data (60S-60N) to the data that has been adjusted by the four factors. The trend of the adjusted data is approximately 27% of the unadjusted GISS LOTI data. In other words, approximately 73% of the rise in global (60S-60N) surface temperature could be natural.
http://i51.tinypic.com/dpvhhd.jpg
Figure 21

And for the sake of discussion, I had EXCEL perform the regression analysis again, but I used “raw” East Indian-West Pacific SST anomalies instead of the detrended data. As one would expect, the multiple regression software created different scaling factors, which are listed in Figure 22. It compares the resulting adjusted global (60S-60N) temperature anomalies to the unadjusted GISS LOTI data. In this instance, the trend of the adjusted data is approximately 18% of the unadjusted data, which is similar to the result I reached using the “eyeball” method and different natural factors in Can Most Of The Rise In The Satellite-Era Surface Temperatures Be Explained Without Anthropogenic Greenhouse Gases?.
http://i51.tinypic.com/2s60p40.jpg
Figure 22

Would the eyeball method with the AMO and East Indian-West Pacific adjustments reduce the global temperature trend by similar amounts? We’ll have to look at that in another post. This one is long enough.

THANKS TO…
…Michael D Smith for suggesting EXCEL to weed out lags, scaling factors, etc. Without his suggestion, I would not have looked for the Analyse-it for Excel. I think I’m going to buy it when the trial period is done.

SOURCES
The GISS LOTI, Reynolds OI.v2 SST (for the NINO3.4 SST anomalies), and HADISST (for the detrended East Indian-West Pacific SST anomalies) data used in this post are available through the KNMI Climate Explorer:
http://climexp.knmi.nl/selectfield_obs.cgi?someone@somewhere

The Stratospheric Aerosol Optical Thickness data is available from GISS:
http://data.giss.nasa.gov/modelforce/strataer/tau_line.txt

And the Atlantic Multidecadal Oscillation data is available from NOAA ESRL:
http://www.esrl.noaa.gov/psd/data/correlation/amon.us.long.data