Hourly INMET data look simple, but three conventions decide whether
daily values computed from them are right: how the hours are stamped,
which day an hour belongs to, and what to do with missing hours. This
vignette explains each one and how inmet_daily() handles
it.
The historical files record time in UTC. inmet_read()
keeps it that way by default, so datetime is unambiguous
for every station in the country:
x$datetime[1:3]
#> [1] "2023-01-01 00:00:00 UTC" "2023-01-01 01:00:00 UTC"
#> [3] "2023-01-01 02:00:00 UTC"
attr(x$datetime, "tzone")
#> [1] "UTC"Many columns describe the hour that ends at the timestamp:
precipitation is the total of the previous hour, and
temp_max_prev_c and temp_min_prev_c are the
highest and lowest air temperature during that hour. The record stamped
03:00 UTC therefore covers 02:00 to 03:00 UTC, which is
23:00 to midnight in Porto Alegre.
Displaying the same instants in local time only changes how they print:
A daily maximum in Brazil should be the maximum of a local
day. Grouping by as.Date(datetime) uses UTC days instead,
which in Porto Alegre run from 21:00 to 21:00 local time (22:00 to 22:00
during daylight saving time, which Brazil observed until 2019). An
evening shower or a hot late afternoon can then land on the wrong
day.
inmet_daily() assigns each record to the local day
containing the middle of its hour (datetime minus 30
minutes). The two choices give different days:
utc_day <- as.Date(x$datetime)
local_day <- as.Date(x$datetime - 1800, tz = "America/Sao_Paulo")
mean(utc_day != local_day)
#> [1] 0.1624149One hour in six (the records stamped 00:00 to 03:00 UTC) is placed on a different day, and the differences matter for daily totals and extremes:
wrong <- tapply(x$precip_mm, utc_day, sum)
right <- inmet_daily(x, min_hours = 1)
right <- setNames(right$precip_mm, right$date)
common <- intersect(names(wrong), names(right))
summary(abs(wrong[common] - right[common]))
#> Min. 1st Qu. Median Mean 3rd Qu. Max.
#> 0.000 0.000 0.000 1.315 0.200 33.200Brazil spans four time zones, and some states observed daylight
saving time in some years and others not. By default
(tz = "auto"), inmet_daily() uses each
station’s own zone, derived from its state and longitude by
inmet_timezone(), and reports it in the
time_zone column:
inmet_timezone(c("RS", "MT", "AC", "AM"), c(-51.2, -56.1, -67.8, -69.9))
#> [1] "America/Sao_Paulo" "America/Cuiaba" "America/Rio_Branco"
#> [4] "America/Eirunepe"Pass a fixed zone, such as tz = "America/Sao_Paulo", to
put all stations on the same clock, or tz = "UTC" for UTC
days.
INMET files contain rows for every hour of the year, including hours
when a station was not reporting. By default inmet_read()
drops rows in which every measurement is missing
(drop_empty = FALSE keeps them). Individual variables can
still be missing in the rows that remain.
A daily total over a day with half its hours missing is not a daily
total. inmet_daily() therefore computes a variable only
when it has at least min_hours valid hourly values that day
(20 by default), and reports the number of hourly records in
n_hours:
daily <- inmet_daily(x)
table(cut(daily$n_hours, c(0, 12, 19, 23, 25), right = TRUE))
#>
#> (0,12] (12,19] (19,23] (23,25]
#> 2 2 46 316
daily[daily$n_hours < 20, c("date", "n_hours", "temp_max_c", "precip_mm")]
#> date n_hours temp_max_c precip_mm
#> 1 2022-12-31 4 NA NA
#> 124 2023-05-03 17 NA NA
#> 148 2023-05-27 1 NA NA
#> 149 2023-05-28 13 NA NATo see completeness before computing anything,
inmet_completeness() gives the percentage of valid hours
per station and month (or year):
head(inmet_completeness(x, variables = c("temp_dry_c", "precip_mm", "wind_speed_ms")))
#> station_code period hours_expected hours_present temp_dry_c precip_mm
#> 1 A801 2023-01 744 734 98.7 98.7
#> 2 A801 2023-02 672 666 99.1 99.1
#> 3 A801 2023-03 744 734 98.7 98.7
#> 4 A801 2023-04 720 720 100.0 100.0
#> 5 A801 2023-05 744 702 94.4 94.4
#> 6 A801 2023-06 720 720 100.0 100.0
#> wind_speed_ms
#> 1 98.5
#> 2 99.1
#> 3 98.7
#> 4 100.0
#> 5 93.7
#> 6 99.7Choose min_hours to suit the variable and the analysis:
precipitation totals are sensitive to every missing hour, while a daily
maximum of temperature is usually reached in the afternoon. Radiation is
an exception: it is missing at night at most stations, so
radiation_mj_m2 sums whatever daytime values are present
(see ?inmet_daily).
The hourly files contain both the air temperature at the top of the
hour (temp_dry_c) and its extremes during the previous hour
(temp_max_prev_c, temp_min_prev_c). The daily
maximum and minimum should come from the extremes; the instantaneous
values miss the peaks between readings:
from_extremes <- inmet_daily(x)$temp_max_c
from_hourly <- inmet_daily(x[, setdiff(names(x), "temp_max_prev_c")])$temp_max_c
summary(from_extremes - from_hourly)
#> Min. 1st Qu. Median Mean 3rd Qu. Max. NA's
#> 0.0000 0.3000 0.5000 0.5707 0.8000 2.3000 4At Porto Alegre in 2023 the hourly readings underestimate the daily maximum by about half a degree on average, and by more than two degrees on some days.
inmet_daily() uses the extremes when they are present
and falls back to temp_dry_c otherwise.
INMET labels automatic-station observations as raw data.
inmet_qc() flags the most common problems: values outside
physical limits, internally inconsistent records (such as an hourly
maximum below the minimum), isolated spikes, and sensors stuck on the
same value. It adds a <variable>_flag column naming
the failed checks, and inmet_daily() and
inmet_completeness() then ignore the flagged values:
Station A801 passed every check in 2023. To see what a flag looks like, plant a sensor spike of 15 degC in one hour:
y <- x
y$temp_dry_c[1000] <- y$temp_dry_c[1000] + 15
flagged <- inmet_qc(y, quiet = TRUE)
flagged[998:1002, c("datetime", "temp_dry_c", "temp_max_prev_c", "temp_dry_c_flag")]
#> datetime temp_dry_c temp_max_prev_c temp_dry_c_flag
#> 998 2023-02-12 02:00:00 24.2 24.6 <NA>
#> 999 2023-02-12 03:00:00 23.6 24.3 <NA>
#> 1000 2023-02-12 04:00:00 38.4 23.8 consistency,spike
#> 1001 2023-02-12 05:00:00 22.7 23.5 <NA>
#> 1002 2023-02-12 06:00:00 21.9 22.6 <NA>The reading is both a spike and inconsistent with the hour’s maximum.
inmet_daily(flagged) leaves it out of the daily statistics,
and attr(flagged, "qc_summary") counts the flags per
variable and check. Across all 545 stations in the 2024 archive the same
checks flag, for example, a stuck wind vane that reported 236 degrees
for months and temperature readings that drop by 9 degC for one hour and
return. Use action = "remove" to set flagged values to
NA directly.
These checks catch gross errors only. Comparing with neighbouring
stations (inmet_nearest()) remains the most effective way
to find subtler problems.