とらりもんHOME  Index  Search  Changes  Login

とらりもん - GPM Diff

  • Added parts are displayed like this.
  • Deleted parts are displayed like this.

!documents
* wikipedia: [[日本語|https://ja.wikipedia.org/wiki/%E5%85%A8%E7%90%83%E9%99%8D%E6%B0%B4%E8%A6%B3%E6%B8%AC%E8%A8%88%E7%94%BB]] [[English|https://en.wikipedia.org/wiki/Global_Precipitation_Measurement]]
* The newest documents are available at: https://www.eorc.jaxa.jp/GPM/archives.html
* Overview of GPM Products Version 2.1 - JAXA/EORC
** https://www.eorc.jaxa.jp/GPM/doc/product/format/ja/01.Overview_of_GPM_Products_J.pdf
** https://www.eorc.jaxa.jp/GPM/doc/product/format/ja/03.GPM_DPR_L2_L3_Product_Format_Documentation_V7_J.pdf
* GPM利用ハンドブック [[https://www.eorc.jaxa.jp/TRMM/documents/PR_algorithm_product_information/doc_pr_v8/GPM_data_util_handbook_V6_20181004_J.pdf]]
* GPM/DPR ATBD (English)
** [2016] https://www.eorc.jaxa.jp/GPM/doc/algorithm/ATBD_DPR_2015_whole_2a.pdf
** [2020] https://www.eorc.jaxa.jp/GPM/doc/algorithm/ATBD_DPR_202006_with_Appendix_a.pdf
** [2023] https://www.eorc.jaxa.jp/GPM/doc/algorithm/ATBD_DPR_L2.pdf

!abbreviation 略語
CSF: classification
DSD: drop size distribution
HS: high sensitivity
PIA: path-integrated attenuation
PRE: precipitation
SLV: solver
SRT: surface reference technique
VER: vertical

!V06AからV07Aへの変更
ファイルフォーマットなど, いろいろ変わったようである。
* KuPRのNSはFSになった!
* KaPRのMSはFSになった!
*資料
** https://www.eorc.jaxa.jp/GPM/doc/algorithm/DPRL2_V07_algorithm_Dec2021_ja.pdf ← P6を参照!
** [[https://gportal.jaxa.jp/gpr/notice/notice/view/1097?lang=ja]]

!Overview



!! Change of Ka-radar scan pattern
On May 21, 2018, the scan pattern (incidence angle) of Ka band radar was changed. This is a big change in the observation mode of GPM satellite. You should be careful about this change, if you treat GPM data!

The change process started at scan 3793 of GPMCOR_KAR_1805211531_1704_024020. Scan 3792 was real value but scan 3793 is NULL value (-9999).

The change process completed at scan 741 of GPMCOR_KAR_1805211704_1837_024021. Scan 740 was NULL value (-9999) and scan 741 is a real value.

! bands and scan patterns
* Ku band (13.6 GHz) ... similar to TRMM/PR (13.796 GHz and 13.802 GHz). for strong rain. Swath 245 km. N66.28deg~S66.30deg.
* Ka band (35.5 GHz) ... no band in TRMM/PR. for weak rain and snow. Swath 125 km
** after 2018/05/21, Ka band will cover entire swath (245 km)
** local zenith angle (degree) of HS: 8.62, 7.86, 7.11, 6.36, 5.60, 4.85, 4.09, 3.34, 2.58, 1.83, 1.07, 0.32, 0.44, 1.19, 1.95, 2.70, 3.46, 4.21, 4.97, 5.72, 6.48, 7.23, 7.99, 8.74
** local zenith angle (degree) of MS: 9.00, 8.24, 7.49, 6.73, 5.98, 5.22, 4.47, 3.71, 2.96, 2.20, 1.45, 0.69, 0.06, 0.82, 1.57, 2.32, 3.08, 3.83, 4.59, 5.34, 6.10, 6.85, 7.61, 8.37, 9.12

incidence angle (in degree) of GPM DPR (sample statistics from level 2 data); 2019/07/08 Kenlo Nishida
KuPR(49) KaPR_MS(25) KaPR_HS(24) ~2018/05/21 KaPR_HS(24) 2018/05/21~
mean stddev step mean stddev step mean stddev mean stddev step
18.186 0.015 9.062 0.017 8.680 0.017 9.760 0.162
17.423 0.014 0.764 8.308 0.019 0.755 7.925 0.008 0.755 10.513 0.184 -0.752
16.667 0.014 0.756 7.556 0.027 0.752 7.174 0.008 0.752 11.271 0.205 -0.758
15.906 0.013 0.761 6.798 0.036 0.758 6.414 0.018 0.760 12.025 0.227 -0.754
15.151 0.012 0.755 6.046 0.045 0.752 5.666 0.028 0.748 12.784 0.249 -0.759
14.391 0.012 0.760 5.292 0.054 0.754 4.911 0.039 0.755 13.535 0.270 -0.752
13.641 0.011 0.750 4.537 0.063 0.756 4.155 0.050 0.756 14.289 0.292 -0.753
12.876 0.010 0.765 3.782 0.072 0.755 3.400 0.061 0.755 15.047 0.313 -0.758
12.120 0.010 0.757 3.030 0.081 0.751 2.650 0.129 0.750 15.809 0.266 -0.762
11.366 0.009 0.753 2.276 0.090 0.755 1.896 0.140 0.754 16.563 0.288 -0.755
10.609 0.009 0.757 1.524 0.099 0.752 1.145 0.151 0.751 17.322 0.309 -0.759
9.852 0.008 0.757 0.769 0.109 0.755 0.388 0.162 0.757 18.078 0.331 -0.756
9.101 0.007 0.751 0.013 0.118 0.757 0.372 0.159 0.016 18.161 0.337 -0.083
8.343 0.007 0.758 0.742 0.103 -0.730 1.125 0.140 -0.754 17.405 0.324 0.756
7.586 0.006 0.757 1.499 0.087 -0.757 1.878 0.122 -0.753 16.648 0.311 0.756
6.835 0.005 0.751 2.245 0.072 -0.746 2.629 0.104 -0.751 15.887 0.298 0.762
6.076 0.005 0.760 3.003 0.057 -0.757 3.386 0.086 -0.757 15.132 0.285 0.755
5.323 0.004 0.752 3.760 0.091 -0.757 4.141 0.068 -0.755 14.373 0.272 0.759
4.567 0.004 0.756 4.512 0.076 -0.753 4.892 0.050 -0.751 13.618 0.259 0.755
3.813 0.003 0.755 5.264 0.060 -0.752 5.644 0.032 -0.751 12.860 0.246 0.757
3.059 0.002 0.754 6.018 0.045 -0.753 6.401 0.015 -0.757 12.107 0.233 0.754
2.305 0.002 0.754 6.774 0.030 -0.757 7.156 0.007 -0.755 11.354 0.220 0.753
1.551 0.001 0.754 7.529 0.016 -0.755 7.908 0.023 -0.752 10.595 0.207 0.758
0.797 0.001 0.754 8.280 0.006 -0.751 8.663 0.041 -0.756 9.842 0.195 0.754
0.046 0.001 0.751 9.034 0.017 -0.754
0.711 0.001 -0.666
1.466 0.001 -0.754
2.219 0.002 -0.753
2.973 0.002 -0.754
3.727 0.003 -0.753
4.481 0.004 -0.754
5.235 0.004 -0.754
5.989 0.005 -0.754
6.747 0.005 -0.758
7.500 0.006 -0.753
8.252 0.007 -0.752
9.010 0.007 -0.758
9.764 0.008 -0.754
10.518 0.008 -0.754
11.274 0.009 -0.757
12.026 0.010 -0.752
12.782 0.010 -0.756
13.547 0.011 -0.764
14.297 0.012 -0.750
15.052 0.012 -0.755
15.809 0.013 -0.757
16.564 0.013 -0.755
17.325 0.014 -0.761
18.082 0.015 -0.757

!products
* GPM KuPR L2 Precipitation, V5A or V5B
* V05A: upto 2018/05/21 0138_0311_024011
* V06A: since 2018/05/21 1704_1837_024021
* for land surface study, use Level 2.
* KaHS data is not included in the G-Portal files (instead, filled by NULL value) after the change of scan pattern

G-Portal (2019/07/02)
for both Ku and Ka,
V06A:  2014/03/08 ~ 2019/06/30

!Download
You can download data from G-portal of JAXA. Use sftp to get large number of files! (You need to set SSH key before it. Check JAXA G-Portal website!)
$ sftp -oPort=2051 [user account]@ftp.gportal.jaxa.jp
> mget standard/GPM/GPM.DPR/2A.Ku/05A/2018/??/??/*h5

!filename rule
example: GPMCOR_KUR_1501010129_0302_004783_L2S_DU2_05A.h5
example: GPMCOR_DPR_1405281041_1214_001397_L2S_DD2_05A.h5
  GPMCOR ... GPM Core satellite (satellite name)
  KUR ... Ku band radar (sensor name)
  140528 ... 2014/05/28 (year/month/day)
  1041_1214 ... 10:41 to 12:14 UT (start time ~ end time)
  001397 ... orbit number
  L2S ... Level2 standard
  DD2 ... algorithm ID
  05A ... product version

!file structure
!! V07A.G-Portal
KuPR
* 最初のデータ: 2014/03/08 22:09 GPMCOR_KUR_1403082209_2342_000144_L2S_DU2_07A.h5
* 1ファイルあたり original: 613 MB。digest(下記): 28 MB
* 全球で1日あたり15.5ファイル。
* 全球で1月あたり465ファイル。
* 全球で1年あたり5660ファイル。
* 全球で2014/03~2023/06 9.3年, 52600ファイル. originalでは32TB

KaPR

* sigmaZeroNPCorrected ... sigma0 corrected for attenuation by the non-precipitation particle
* VER: Vertical Profile Module
* SLV: Solver module
* SRT: surface reference technique (SRT)

陸面解析に不要そうなDatasetを削除して小さなファイルにするスクリプト
#!/usr/bin/python3
import h5py
import sys
# Define source and destination file paths
if len(sys.argv) != 3:
    print("Usage: ./GPMKuPR_make_digest.py [source file] [destination file]")
    sys.exit(1)
src_file, dst_file = sys.argv[1], sys.argv[2]
# List of datasets to exclude.
#   There are 131 datasets in one HDF5 file. If extracted in raw data, their total size is 3103064503 bytes.
#   Within them, the following 20 datasets (total size in raw data is 2870594048 bytes; 92.5%) are selected for deleting as a criteria of bigger than 1555456 byte, no flag, no sigmaZero.
exclude_datasets = [
   "FS/DSD/binNode",
   "FS/DSD/paramRDm",
   "FS/DSD/phase",
   "FS/PRE/height",
   "FS/PRE/zFactorMeasured",
   "FS/SLV/epsilon",
   "FS/SLV/paramDSD",
   "FS/SLV/paramNUBF",
   "FS/SLV/precipRate",
   "FS/SLV/precipWater",
   "FS/SLV/zFactorFinal",
   "FS/SRT/PIAalt",
   "FS/SRT/PIAweight",
   "FS/SRT/RFactorAlt",
   "FS/SRT/refScanID",
   "FS/SRT/stddevEff",
   "FS/VER/airTemperature",
   "FS/VER/attenuationNP",
   "FS/VER/piaNP",
   "FS/VER/piaNPrainFree"
]

# Open source file
src = h5py.File(src_file, 'r')
# Open the destination file
dst = h5py.File(dst_file, 'w')

def copy_item(name, item):
    if name in exclude_datasets:
#        print (name, " will be excluded!")
        return  # Skip the excluded datasets
    if isinstance(item, h5py.Dataset):
        # Copy dataset
        dst.create_dataset(name, data=src[name], compression="gzip", compression_opts=9)
    elif isinstance(item, h5py.Group):
        # Copy group
        dst.create_group(name)
        
# Walk through source file and copy each item to the destination file
src.visititems(copy_item)
# Close the files explicitly
src.close()
dst.close()


!! V06A.G-Portal
/NS ... KuPR
/HS ... KaPR "high-sensitivity"
/MS ... KaPR "matched scan" (Same position as KuPR)
/NS/PRE/sigmaZeroMeasured ... for land monitoring
/NS/PRE/localZenithAngle ... local incident angle
/NS/PRE/flagPrecip ... precipitation flag; 0 for no precipitation, 1 for precipitation
/NS/PRE/landSurfaceTyp ... ocean mask; 0 - 99 Ocean, 100 - 199 Land, 200 - 299 Coast, 300 - 399 Inland water
/NS/FLG/qualityData ... quality flag
/NS/Latitude ... latitude
/NS/Longitude ... longitude

!How to handle GPM data in python
import h5py
fn="GPMTRM_PR1_0508301259_1432_044399_L2S_PU2_8b18_mod.h5"
infh=h5py.File(fn, 'r')
infh.keys()
zc=infh['/NS/PRE/localZenithAngle'].value
for i in range(za[1,:].size):
            print([i, np.mean(zc[:,i]), np.std(zc[:,i])])