GOVST IV-TT meeting - godae.orggodae-data/GOVST-IV/presentations/TT-day/2.IV...Proposed verification...

GOVST-III Meeting, Rio de Janeiro, 5-9 November 2012

GOVST IV-TT meeting

5th November 2012

Rio de Janeiro


Agenda, scope

• Class 4 intercomparison

• Multi-model ensemble

• GSOP2012 intercomparison

• Other


Class 4 intercomparison


Background + goals • A verification intercomparison

• Following similar methodology to JCOMM wave

intercomparison:

– share model counterparts in observation space (cls4)

– any centre can download and intercompare results

– one centre (UKMO) computes statistics and makes plots

available to participants

– results not released further without the permission of all

participants

• If successful, should be adopted by JCOMM as “official”

intercomparison methodology for OOFS


1. Class 4 inter-comparison

Overview

UKMO download obs files and produce

model analysis, forecast, persistence

and climatological information at the obs

locations in netCDF format.

US GODAE server

Each OOFS downloads the UKMO files

and replaces the model data with those

from their system.

Any participating group can download

files and inter-compare the results.

Produce a core set of plots for the GOV

web-site (password protected).

Observation inputs

~5 days behind real time


Present status • 3 centres contributing class 4 files in near real

time: UKMO, BLUELink, NCEP (profiles)

• Obs are:

– SST – surface drifters from USGODAE (+ AATSR)

– profiles – Argo from Coriolis/MyOcean (5 day delay)

– SLA – Jason-1+2 + Cryosat from Aviso (+ UKMO obs

processing)

• UKMO production not yet operational - some

gaps in the record

• Demonstration intercomparison results produced

by UKMO – participants now checking these


Experiences / feedback from

BLUELink, NCEP, REMO


Preliminary results • As a demonstration of what could be shared

• These plots are anonymised because only

participants get to see results

• Plots are for profiles only (3 participants)

• Important caveats:

– Very short period (~2 weeks) so may not be

representative

– Participants have not yet checked the results, so

there could be mistakes

– These simple metrics do not tell the whole story


RMSE Bias

Day 1 Day 5


Anomaly correlation

Day 1 Day 5


Best estimate Day 5


RMSE Bias

Forecast Persistence

Climatology


Title

Climatology


1. Class 4 inter-comparison

Prototype web pages


Issues + next steps

• Robustness of UKMO production

• Participants to verify results / method

• Other participants to join

• Verification areas (proposal on next slide)

• Post-processing “rules” (esp. SLA)

• More observation types (longer term?):

– satellite SST (SLSTR, or sooner?)

– currents (drifters?)

– sea-ice

• More advance verification methods


GODAE class 1 regions

Overlaps

causing

problems for

interpretation of

verification

results – e.g.

large Tropical

Pacific errors

contaminate

North and

South Pacific

stats


Proposed verification regions

TAT

NPA

TPA

ARC

NAT NPA

TPA

IND

SPA SAT

ACC

MED

SPA


Proposed verification regions:

non-overlapping (mostly)

Name

Short

name

Latitude

range

Longitude

range

Excluded basins

North Atlantic NAT 20 - 66N 100W - 31E Mediterranean, Pacific

South Atlantic SAT 60 - 20S 70W - 20E

Tropical Atlantic TAT 20S - 20N 90W - 15E

North Pacific NPA 20 - 66N 100E - 77W

Atlantic, Gulf of Mexico,

Caribbean Sea

South Pacific SPA 60 - 20S 150E - 70W

Tropical Pacific TPA 20S - 20N 90E - 70W

Caribbean Sea,

Area south of Indonesia

and west of 120E

Indian Ocean IND 40S - 31N 20E - 120E

Gulf of Thailand and area

north of Indonesia

Arctic Ocean ARC 66 - 90N 180W - 180E

Southern

Ocean ACC 89S - 35S 180W - 180E

Mediterranean MED 30N - 48N 6W - 42E Atlantic

Global GLO 89S - 90N 180W - 180E

Others (specific to participants: Australia, Brazil, ...) [100m isobath?]


Multi-model ensemble

demonstrations


Multi-model ensemble

demonstration • Participating centres make surface fields available for

download

– native grid if possible

– daily mean

– 4 centres contributing (NCEP, NRL, Bluelink, UKMO)

• 2 partners proposed to produce demonstrations (NCEP-

global, CSIRO-Australia)

– Results from partners: NCEP, Mercator, ...

• Lessons learned?

• Ideas for 2013...


Other topics


Updates to membership?

BoM, Australia Gary Brassington, Prasanth Divakaran

CSIRO, Australia Peter Oke

Environment Canada Hal Richie, Greg Smith

INGV, Italy Marina Tonani

Mercator-Ocean, France Fabrice Hernandez (co-chair)

NERSC, Norway Pavel Sakov

NMEFC, China Liying Wan

NOAA-NCEP, USA Avichal Mehra, Todd Spindler

NRL, USA Jim Cummings

UFBA, Brazil Clemente Tanajura

UK Met Office Alistair Sellar (co-chair), Matt Martin


WWRP/WGNE Joint Working Group

on Forecast Verification Research

• Approximately the NWP equivalent of IV-TT

• Very active in exploring new verification methods

• Their remit includes promotion of good practice in

verification (as does ours)

• Can we collaborate / piggy-back to help achieve some of

our aims?

• 2 initial ideas:

– Raise awareness of (and contribute to) the very good CAWCR

verification website

– JWGFVR run verification tutorial workshops. Explore the

possibility of opening these up to ocean forecasting students,

and/or borrowing some material for GODAE summer schools.

http://www.cawcr.gov.au/projects/verification/

http://www.cawcr.gov.au/projects/verification/

GOVST IV-TT meeting - godae.orggodae-data/GOVST-IV/presentations/TT-day/2.IV...Proposed verification...

Documents

Transcript of GOVST IV-TT meeting - godae.orggodae-data/GOVST-IV/presentations/TT-day/2.IV...Proposed verification...