How to get and use PEN data?

Under the data policy, you can get and use PEN data.

Data servers

The PEN data can be dowloaded via internet. They are distributed by two servers:

We believe that the JAMSTEC server is more robust and well controlled by network engineers, hence it would be better to try JAMSTEC server for downloading a big amount of data. Meanwhile, the update of the new data and information would be more quick and detailed in the PEN mother server.


Data structure and naming conventions

The PEN data are organized by site-basis. It means you access to the data site-by-site. For example, if you want data taken in TKY site (Takayama Site in Japan), you access to the directory of TKY site. The directory tree of each site is as follows:
├── original
│   ├── dc     (original data of ADFC ... timelapse camera images)
│   ├── sp     (original data of sun-photometer)
│   └── sr     (original data of HSSR ... spectroradiometers)
└── summary
    ├── dc     (summary data of ADFC ... extracted and reduced in size)
    ├── sp     (perhaps no data)
    └── sr     (summary data of HSSR ... converted to txt and reflectance, etc.)
The "summary" directories are open to public (basically), but some of the "original" directories may be closed because of specific data policies.

If a site is not equipped with some sensors (such as sr or sp), those corresponding directories do not exist. Because most sites are equipped with ADFC, they have original/dc and summary/dc directories, but other directories may not exist for some sites. Especially "sp" directories are uncommon because sun-photometers are installed in only a few sites.

The sub-directories under the above mentioned directories are like this:

├── original
│   ├── dc
│   ├── dc_2004
│   │   ├── dc_2004_001
│   │   │   ├── dc_2004_001_1152+0900_TKY__y18_d.jpg
│   │   │   ├── dc_2004_001_1207+0900_TKY__y18_e.jpg
│   │   │   └── ...
│   │   ├── dc_2004_002
│   │   │   ├── dc_2004_002_1159+0900_TKY__y18_d.jpg
│   │   │   ├── dc_2004_002_1216+0900_TKY__f02_d.jpg
│   │   │   └── ...
│   │   ...
│   │   
│   ├── dc_2005
│   │   ├── dc_2005_001
│   │   ...
│   ├── dc_2006
│   │   ├── dc_2006_001
│   │   ...
Basically, by going deeper in the directory trees, the names of directories and files become longer by taking the names of the previous directories. The convention of the naming is:
AB_CDEF_GHI_JKLM+NOPQ_RSTU_VWXYZ.###
	AB:   sensor type (dc, sp, sr)
	CDEF: year
	GHI:  date (Julian Date, starting from 001 on January 1st and ending at 365 or 366 on december 31.)
	JKLM: hour and minutes
	NOPQ: time zone (GMT+####)
	RSTU: site ID
	VWXYZ:sensor ID (3 letters for location, 2 for function or target or direction)
	###:  extention (jpg, rad, txt etc)

example: dc_2004_003_0400+0900_TGF__btp_u.jpg
	ADFC (automatic digital fisheye camera) image, year 2004, day 003 (January 3rd), 
        04:00 in the time Zone GMT+09:00 (Japan Standard Time), site is TGF_ (TERC grass field), 
        btp_u (building top, upword ... watching the sky)
For more detail, please check the following review and data papers:
Nasahara KN, Nagai S (2015) Development of an in-situ observation network for terrestrial ecological remote sensing — the Phenological Eyes Network (PEN). Ecological Research, DOI 10.1007/s11284-014-1239-x (open access)

Nagai S, Akitsu T, Saitoh TM, Busey RC, Fukuzawa K, Honda Y, Ichie T, Ide R, Ikawa H, Iwasaki A, Iwao K, Kajiwara K, Kang S-K, Kim Y-W, Khoon KL, Kononov AV, Kosugi Y, Maeda T, Mamiya W, Matsuoka M, Maximov TC, Menzel A, Miura T, Mizunuma T, Morozumi T, Motohka T, Muraoka H, Nagano H, Nakai T, Nakaji T, Oguma H, Ohta T, Ono K, Pungga RAS, Petrov RE, Sakai R, Schunk C, Sekikawa S, Shakhmatov R, Son Y-H, Sugimoto A, Suzuki R, Takagi K, Takanashi S, Tei S, Tsuchida S, Yamamoto H, Yamasaki E, Yamashita M, Yoon TK, Yoshida T, Yoshimura M, Yoshitake S, Wilkinson M, Wingate L, Nasahara KN (2018) 8 million phenological and sky images from 29 ecosystems from the Arctic to the tropics: the Phenological Eyes Network. Ecological Research, https://doi.org/10.1007/s11284-018-1633-x

The ID of cameras (location, direction, etc.) are described in detail in the Nagai et al. (2018) paper.