Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Emulating Rainfall–Runoff-Inundation Model Using Deep Neural Network with Dimensionality Reduction

Emulating Rainfall–Runoff-Inundation Model Using Deep Neural Network with Dimensionality Reduction JANUARY 2023 MO M O I E T A L. 1 Emulating Rainfall–Runoff-Inundation Model Using Deep Neural Network with Dimensionality Reduction a,b c,d,e,f g,b g h MASAHIRO MOMOI , SHUNJI KOTSUKI, RYOTA KIKUCHI, SATOSHI WATANABE, MASAFUMI YAMADA, AND SHIORI ABE GRASP SAS, Lezennes, France DoerResearch, Inc., Nagoya, Japan Institute for Advanced Academic Research, Chiba University, Chiba, Japan Center for Environmental Remote Sensing, Chiba University, Chiba, Japan RIKEN Center for Computational Science, Kobe, Japan RPRESTO, Japan Science and Technology Agency, Chiba, Japan Office of Society Academia Collaboration for Innovation, Kyoto University, Kyoto, Japan Disaster Prevention Research Institute, Kyoto University, Kyoto, Japan Mitsui Consultants Co., Ltd., Tokyo, Japan (Manuscript received 23 May 2022, in final form 14 November 2022) ABSTRACT: Predicting the spatial distribution of maximum inundation depth (depth-MAP) is important for the mitigation of hydrological disasters induced by extreme precipitation. However, physics-based rainfall–runoff-inundation (RRI) models, which are used operationally to predict hydrological disasters in Japan, require massive computational resources for numerical simulations. Here, we aimed at developing a computationally inexpensive deep learning model (Rain2Depth) that emulates an RRI model. Our study focused on the Omono River (Akita Prefecture, Japan) and predicted the depth-MAP from spatial and temporal rainfall data for individual events. Rain2Depth was developed based on a convolutional neural network (CNN) and predicts depth-MAP from 7-day successive hourly rainfall at 13 rain gauge stations in the basin. For training the Rain2Depth, we simulated the depth-MAP by the RRI model forced by 50 ensembles of 30-yr data from large-ensemble weather/climate predictions. Instead of using the input and output data directly, we extracted important features from input and output data with two dimensionality reduction techniques [principal component analysis (PCA) and the CNN approach] prior to training the network. This dimensionality reduction aimed to avoid overfitting caused by insufficient training data. The nonlinear CNN approach was superior to the linear PCA for extracting features. Finally, the Rain2Depth architecture was built by connecting the extracted features between input and output data through a neural network. Rain2Depth-based predictions were more ac- curate than predictions from our previous model (K20), which used ensemble learning of multiple regularized regressions for a specific station. Whereas the K20 can predict maximum inundation depth only at stations, our study achieved depth-MAP pre- diction by training only the single model Rain2Depth. KEYWORDS: Flood events; Runoff; Deep learning; Dimensionality reduction 1. Introduction The ensemble NWP forecasts enable probabilistic flood fore- casts in which ensemble NWP forecasts are used as input for Recent advances in high-performance computers (HPCs) have ensemble hydrological simulations. In the past two decades, enabled numerical weather prediction (NWP) at high spatial res- extensive hydrological studies have been advancing knowledge olution and with ensembles of .1000 members. For example, of the physical processes of river discharge, runoff, and two- Yashiro et al. (2020) successfully conducted a global atmospheric dimensional inundation (e.g., Sayama et al. 2012; Yamazaki data assimilation experiment at 3.5-km resolution with 1024 et al. 2011). Recent studies carry out probabilistic flood forecasts ensembles using the Fugaku, the flagship supercomputer of using ensemble NWP data (e.g., Kobayashi et al. 2020). Owing to Japan. In the past decade, the computational resources of HPCs the progress of physical-based models, realistic simulation can be have been increased by adding cores of the central processing conducted for an operational flood warning system. However, unit. For such many-core HPCs, increasing the number of ensem- the expanded physical-based models require progressively more ble members of NWP models is more scalable than reducing the and more computational resources, as observed in NWP models. horizontal and vertical resolutions because ensemble forecasts For real-time flood warning systems, the exploration of computa- can be essentially parallelized. Therefore, a Japanese project is tionally inexpensive inundation prediction methods is an impor- intended to prevent and mitigate weather-related disas- tant alternative to enhanced ensemble NWPs. For that purpose, ters through the effective use of large-ensemble weather this study uses deep neural network techniques. predictions. Recently, applications of deep learning in NWP have been massively investigated. One application is to use deep learning for data-driven weather prediction such as precipitation nowcast- ing (e.g., Shi et al. 2017; Ravuri et al. 2021). Also, using deep learning in the postprocessing step to adjust model outputs Corresponding author: Masahiro Momoi, momoi-masahiro@ doerresearch.com is also known to be beneficial (e.g., Gronquist et al. 2021; DOI: 10.1175/AIES-D-22-0036.1 e220036 Ó 2023 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses). 2 AR TI F I C I A L I N T E LLI G E N C E F O R T H E E AR TH S Y S T E M S VOLUME 2 Hess and Boers 2022). Combining deep learning and data flooded rivers in Japan; we compare the result with the physical- assimilation is also well-suited to applications such as learn- based model simulation. ing data assimilation (e.g., Bocquet et al. 2021; Tsuyuki and This study is organized as follows. Section 2 describes Tamura 2022) and learning observation operators (e.g., Liang the methods and experimental settings. Section 3 reveals et al. 2021). Challenging studies aim at emulating NWP models the experimental results and provides discussion. Section 4 for learning relations between input and output weather data provides a summary. (Pathak et al. 2022; Keisler 2022). Namely, these studies use deep learning for emulating meteorological predictions, model 2. Methods/experimental design output adjustments, data assimilation, observation operator, This study proposes a machine learning model, called and atmospheric dynamical processes. Rain2Depth, that computes the spatial distributions of max- The emulation of physical-based models using neural net- imum inundation depth from input rainfall data (Fig. 1). works has been also investigated broadly in the Earth sciences. Training inundation data for Rain2Depth were generated For example, in atmospheric radiation studies, progress to- by a physical-based rainfall–runoff-inundation (RRI) model ward addressing the lack of computational resources has been (Sayama et al. 2012), an operational real-time flood prediction made by moving from physical-based to machine learning– model used in Japan (e.g., in Hyogo Prefecture). The Policy based models. Takenaka et al. (2011) and Shi et al. (2020) re- Decision Making for Future Climate Change (d4PDF; Mizuta constructed the output (i.e., sky radiance and radiative flux) of et al. 2017) database was used for the input rainfall data, as the radiative transfer model; quasi-real-time processing of described in section 2a. The Rain2Depth model is based on a satellite observations was successfully achieved (e.g., neural network (section 2c). To enhance generality, we applied Hashimoto and Nakajima 2017). Inspired by these previous dimensionality reduction techniques to input and output data studies, we aim to develop a deep learning–based inundation prior to neural network training (section 2b). The models were prediction method, called Rain2Depth, that emulates the cali- evaluated with cross validation, as described in section 2d. brated physical-based inundation model. In the previous stud- ies, most of the emulators of the inundation model have been a. Training data developed for emulating the time series of the inundation 1) TARGET RIVER BASIN:OMONO RIVER BASIN depth using rainfall successively and can be classified into the following two types: emulating at a particular site (e.g., Mosavi The Omono River, with a basin area of 4710 km , is a class- et al. 2018) and in the area spatially (e.g., Chang et al. 2010; A river located in the northeastern part of Japan. It is one of Lin et al. 2013; Jhong et al. 2017). The study on the emulator the most frequently flooded rivers in Japan; it has been of a particular site has been investigated through more than flooded four times since 2000. Therefore, the development of 100 papers using various machine learning techniques in the rapid flood forecasting systems is critical in this region. Historical last 2 decades (Mosavi et al. 2018). floods have mainly been caused by the stationary baiu front. In particular, we aimed to develop a deep learning–based surrogate model of the state-of-the-art inundation model, es- 2) RAINFALL DATA OF D4PDF WITH BIAS CORRECTION pecially for evacuation plans in the early stages of rainfall. As Rainfall data obtained from d4PDF (Mizuta et al. 2017) compared with the abovementioned relevant studies, this were used as input for RRI simulations. The d4PDF provides study focuses on the maximum inundation depth during the the results of ensemble experiments that comprise more than event. Kotsuki et al. (2020) proposed a regression-based emu- 1000 years of meteorological data for both historical reproduc- lator of an inundation model and demonstrated good agree- tions and future projections. Specifically, the historical experi- ments with model-based predictions. However, the machine ments reproduced the past period of 1951–2010 with 20-km- proposed by Kotsuki et al. (2020) can only predict inundation resolution 50 ensembles, which was dynamically downscaled by depth at stations. Therefore, we proposed a new machine that 60-km global ensemble experiment driven by perturbed bound- can predict the spatial pattern of inundation depth using a ary conditions to sea surface and sea ice temperature. This study time series of distributed rainfall data. General deep neural used these 50 ensemble experiments, with a focus on the past networks, including networks in the aforementioned studies 30 years (1981–2010). We identified the event with the maxi- (Takenaka et al. 2011; Shi et al. 2020), require very large mum 30-h precipitation in each year. For the RRI simulations, amounts of training data to optimize massive parameters in we extracted successive 168-h (i.e., 7 day) precipitation data the network. In general, the high-resolution inundation simu- including the heaviest precipitation: 48-h spin up, 30-h heavy lation requires high-dimensional input data and produces precipitation, and 90-h rest periods. In total, 1500 events were high dimensional output data. However, for hydrological em- extracted for RRI simulation. ulators, there would be fewer essential features within input Prior to the RRI simulation, a bias correction method rainfall data and output inundation patterns. Here, we pro- pose training the network with features extracted by linear (Watanabe et al. 2020) was applied to the d4PDF rainfall data principal component analysis and the nonlinear neural for reducing unignorable bias in model-based precipitation network–based autoencoder (section 3a). We then build the data. The observation dataset of the Automated Meteorological Rain2Depth model by connecting input and output features, Data Acquisition System (AMeDAS) operated by the Japan as described in section 3b. In this study, we apply the proposed Meteorological Agency was used as reference data for the bias machine to the Omono River, one of the most frequently correction. This bias correction led to improved extreme JANUARY 2023 MO M O I E T A L. 3 FIG. 1. Conceptual design of this study; input data were the rainfall data generated from d4PDF with the bias correction method (Watanabe et al. 2020) described in section 2a(2); reference output data were the spatial distribution of maximum inundation depth generated from the rainfall data with the physical-based RRI model (Sayama et al. 2012) described in section 2a(3); K20 and Rain2Depth are the emula- tors developed by Kotsuki et al. (2020) and this study, respectively. Points A and B in the figure are the target sites for Kotsuki et al. (2020). precipitation events in the past. The operational design rainfall channels surveyed by the Ministry of Land, Infrastructure, Transportation and Tourism of Japan. In addition, we mod- of the Omono River, which is 258.7 mm per 2-day period, was reproduced with an error of ,10% using this bias correction eled the flood control operations of six dams in the Omono for the d4PDF. In this procedure, the rainfall data were discre- River (Konja et al. 2018). To model runoff processes, we ap- tized spatially into 13 AMeDAS stational data of 168 h. plied the unsaturated lateral flow mechanism for forest area and the saturated lateral flow mechanism for other land-use areas. The model was calibrated with two heavy rainfall events 3) RRI MODEL AND MAXIMUM INUNDATION DEPTH that occurred in 2004 and 2011; it was validated by focusing on The RRI model is a two-dimensional physical-based distrib- the record-breaking flood in 2017. The RRI model showed uted hydrological model, that simultaneously simulates both the good reproducibility in terms of the observed discharge, inun- rainfall–runoff process in slope areas and the flood inundation dation area and depth (Abe et al. 2019). For all six official dis- process in rivers and floodplains. In this section, we first describe charge observation points, Nash–Sutcliff efficiency values the RRI model setting followed by experimental setting to pro- were .0.70. The reproducibility of inundated and noninun- duce spatial inundation depth training data. dated areas was 90%. See Abe et al. (2019) for more details. This study used the RRI model calibrated and validated by Using the calibrated RRI model, we conducted experi- Abe et al. (2019) in the Omono River. The spatial resolution ments with the bias-corrected d4PDF rainfall data for 1500 of the model is 270 m. All grid cells with an upstream area of events. We used the maximum inundation depth of each grid .2km were regarded as river grid cells. For the river geometry to generate the spatial distribution of maximum inundation of main rivers, we included the cross-section shapes of river depth (depth-MAP) for each event. 4 AR TI F I C I A L I N T E LLI G E N C E F O R T H E E AR TH S Y S T E M S VOLUME 2 TABLE 1. Architecture of the convolutional autoencoder for rainfall data. Here and in subsequent tables, the numbers in brackets indicate the dimensions of the array in the program. Layer type Output channel Kernel size Padding Stride Encoder (Input shape: [13, 1, 168]) Conv1d 1 ReLU [13, 16, 168] 3 1 1 Conv1d 1 ReLU [13, 16, 168] 3 1 1 Conv1d 1 ReLU [13, 16, 168] 3 1 1 Conv1d 1 ReLU [13, 16, 168] 3 1 1 Conv1d 1 ReLU [13, 16, 84] 2 0 2 Conv1d 1 ReLU [13, 16, 84] 3 1 1 Conv1d 1 ReLU [13, 16, 84] 3 1 1 Conv1d 1 ReLU [13, 16, 28] 3 0 3 Conv1d 1 ReLU [13, 16, 14] 2 0 2 Linear 1 LayerNorm [13, 81] }} } Decoder (Input shape: [13, 81]) Linear [13, 224] }} } ConvTranspose1d 1 ReLU [13, 16, 28] 2 0 2 ConvTranspose1d 1 ReLU [13, 16, 84] 3 0 3 Conv1d 1 ReLU [13, 16, 84] 3 1 1 Conv1d 1 ReLU [13, 16, 84] 3 1 1 ConvTranspose1d 1 ReLU [13, 16, 168] 2 0 2 Conv1d 1 ReLU [13, 16, 168] 3 1 1 Conv1d 1 ReLU [13, 16, 168] 3 1 1 Conv1d 1 ReLU [13, 16, 168] 3 1 1 Conv1d [13, 16, 168] 3 1 1 b. Dimensionality reduction techniques The present study designed the AE as follows. The CNN-AE for input rainfall data was constructed by one-dimensional con- Deep neural network training processes require massive volution (Conv1d). Conv1d was developed for electrocardiogram training data because such processes involve large numbers of classification (Kiranyaz et al. 2015). Recent studies have used network parameters depending on the features of the input Conv1d for time series data in geophysics (e.g., Makinoshima and output data. However, this study endeavored to train the et al. 2021; Van et al. 2020) because of its low computational cost network with a moderately small number of events (1500) as (Kiranyaz et al. 2021). For rainfall–runoff emulation, Conv1d training data. Therefore, it was beneficial to extract important may bemoresuitablethan longshort-term memory(Hochreiter features from input (rainfall) and output (depth MAP) data, and Schmidhuber 1997) because Conv1d effectively extracts the then use these extracted features to train a network with dependencies (features) in short-term time series (Van et al. reduced network parameters. Here, we extracted important 2020). Therefore, the features of the rainfall data were extracted features by reducing the data dimensionality. using the Conv1d, fully connected layer (“Linear” in the tables), The easiest way to reduce data dimensionality is principal com- rectified linear units (ReLU; Nair and Hinton 2010), and layer ponent analysis (PCA); this method reduces the dataset features normalization (“LayerNorm” in the tables; Ba et al. 2016), as after orthogonalization by singular value decomposition. PCA is shown in Table 1. The CNN-AE for the rainfall data was con- known as statistical recognition tools as empirical orthogonal structed using the one-dimensional convolutional layers and a functions (Wu et al. 2009) or proper orthogonal decomposition fully connected layer in the encoder, while one-dimensional (Lumley 1967) in meteorology and geophysical fluid dynamics transposed convolutional layers and a fully connected layer fields, for example, postprocessing tools (Murray and Ukeiley were used in the decoder. Layer normalization can constrain 2007) and dominant components analysis (Kikuchi et al. 2016). the network parameters by specific layer data without batch However, PCA would be suboptimal choice if major modes are data (Ba et al. 2016). Therefore, it is applicable for data with nonlinear. An alternative method is a neural network–based both large and small batch sizes. Layer normalization is also technique known as autoencoder (AE), which can extract fea- suitable for large variance data, such as the extreme weather tures through nonlinear multilayer neural networks with activa- data used in this study; other normalization techniques (e.g., tion functions. In this study, we used the convolutional–neural batch normalization; Ioffe and Christian 2015) are not suitable network (CNN) AE (CNN-AE). It uses CNN networks as for large variance data. encoder and decoder before and after feature extraction using a Two-dimensional convolutional neural networks for image fully connected layer. Because the networks (e.g., activation data are rapidly progressing technologies (e.g., Krizhevsky function, normalization, and layer numbers) were built empiri- et al. 2012). A two-dimensional convolutional layer (Conv2d) cally using training and validation (TRAIN/VAL) data, there convolves the neighbor pixels of a target pixel and, thus, poten- might still be room for optimization. This study compared the tially extracts spatially distributed local features. These character- efficiency of data reduction between PCA and AE methods. istics would be beneficial for the feature extraction of depth-MAP JANUARY 2023 MO M O I E T A L. 5 TABLE 2. Architecture of the convolutional autoencoder for depth-MAP. Layer type Output channel Kernel size Padding Stride Encoder (Input shape: [1, 1, 432, 324]) Conv2d 1 ReLU [1, 128, 216, 162] 2 0 2 Conv2d 1 ReLU [1, 128, 108, 81] 2 0 2 Conv2d 1 ReLU [1, 128, 36, 27] 3 0 3 Conv2d 1 ReLU [1, 128, 12, 9] 3 0 3 Conv2d 1 ReLU [1, 128, 4, 3] 3 0 3 Linear 1 LayerNorm [1, 10] }} } Decoder (Input shape: [1, 10]) Linear [1, 1536] }} } ConvTranspose2d 1 ReLU [1, 128, 12, 9] 3 0 3 ConvTranspose2d 1 ReLU [1, 128, 36, 27] 3 0 3 ConvTranspose2d 1 ReLU [1, 128, 108, 81] 3 0 3 ConvTranspose2d 1 ReLU [1, 128, 216, 162] 2 0 2 ConvTranspose2d [1, 1, 432, 324] 2 0 2 because the inundated area should be continuously distributed trials. For the validation of the small set of training data, the around the river. In this study, we constructed the CNN-AE for score of the network was determined from the evaluation depth-MAP data by using a two-dimensional convolutional score of each trial derived from subtrials (Fig. 2b). For example, neural network (Table 2). when 400 data were used for training, the score of each trial was derived from 6 [i.e., combination C(4, 2)] subtrials. c. Neural network architectures e. Brief outline of an emulator with multiple To emulate the maximum inundation depth from rainfall data regularized regressors (i.e., Rain2Depth), we constructed a neural network by connect- ing the features of rainfall and the depth-MAP data extracted by Kotsuki et al. (2020) emulated the maximum inundation dimensionality reduction techniques, as described in section 2b. depth at the two locations denoted as points A and B in Fig. 1 The network consisted of the one-dimensional convolutional net- through ensemble learning with multiple regularized regres- work, ReLU, and the fully connected layer, as shown in Table 3. sors (herein this method is referred to as K20). The architec- ture of their emulator is illustrated in Fig. 1. It consists of the d. Cross validation following two aspects: maximum inundation depth at the spe- The dataset was grouped into 1000 data for training (labeled cific location predicted by three regressors with different regu- as TRAIN)/validation (labeled as VAL) and 500 data for tests larization [ridge, least absolute shrinkage and selection (labeled as TEST), as shown in Fig. 2.TEST data were not operator (LASSO), and elastic net] and ensemble learning us- used for emulator validation and architecting, as detailed in ing a random-forest classifier. In this study, K20 was trained sections 3a and 3b(1). After the network had been architected using TRAIN/VAL data (1000 data) as described by Kotsuki [section 3b(2)], the pseudo generalization performance of the et al. (2020) with the hyperparameters of regularization deter- network was estimated from TEST data that consist of inde- mined in their paper. pendent data from the TRAIN/VAL training procedure. Vali- dations of the models were performed via fivefold cross 3. Results and discussion validation (5FCV) using TRAIN/VAL and a small training da- taset that allowed assessment of training model feasibility. The a. Dimensionality reduction 5FCV divides the TRAIN/VAL into five subsets; it uses four 1) RAINFALL DATA subsets (total of 800 data) for TRAIN and the remaining sub- set (200 data) for VAL (Fig. 2a). Therefore, the evaluation Because the rainfall data of 13 AMeDAS stations con- score of the network was determined from the mean of five tained hourly data over seven days, the rainfall data had two TABLE 3. Architecture of the convolutional neural network of the middle layer in Rain2Depth. Layer type Output channel Kernel size Padding Stride (Input shape: [1, 81, 13]) Conv1d 1 ReLU [1, 200, 13] 1 0 1 Conv1d 1 ReLU [1, 200, 13] 1 0 1 Conv1d 1 ReLU [1, 200, 13] 1 0 1 Conv1d 1 ReLU [1, 200, 13] 1 0 1 Conv1d 1 ReLU [1, 20, 1] 13 0 1 Linear [1, 10] }} } 6 AR TI F I C I A L I N T E LLI G E N C E F O R T H E E AR TH S Y S T E M S VOLUME 2 FIG.2.Concept of fivefold cross validation: (a) general description of 5FCV and (b) description for small numbers of data points. dimensions and 2184 variables (168 h 3 13 AMeDAS stations) suggesting that the dimension of the rain-2D data is signifi- in each event (herein, rain-2D data). This number exceeded cantly increased beyond 800 data. Therefore, additional train- the number of training data; thus, we reduced dimensionality ing data for dimensionality reduction are needed to retain by using PCA. The dashed lines in Fig. 3 show the PCA results generalization ability. This feature arises from two factors: the for 200–800 training data. The root-mean-squared errors temporal distribution of rainfall (i.e., time series of rainfall) (RMSEs) of TRAIN and VAL decreased as the number of and spatial distribution of rainfall of the 13 AMeDAS stations training data increased; the use of 800 data for training yielded (e.g., change in rainfall region position from south to north or the best performance. An increased number of principal com- another movement). If the features of the temporal distribu- ponents was expected to improve the performance of TRAIN tion of rainfall are identical at all AMeDAS stations, the rainfall data reconstruction, but the TRAIN rainfall data did dimension of the rainfall data observed at each AMeDAS not match the VAL rainfall data because of overfitting. When station (herein, rain1d) can be reduced with the same machine. 200 training data were used, the VAL performance was not This assumption also produces 13-fold more effective training significantly improved with respect to TRAIN performance data relative to rain-2D data. The dimension of the rain-1D when the number of principal components was .10. These data can be reduced while retaining generalization perfor- findings suggest that principal components beyond the 10th mance even when using 200 training data (solid lines in Fig. 3). component do not enhance generalization performance. These Thus, the features of the temporal distribution of the rainfall findings were also observed for principal components beyond (such as rain and stop mechanism) are similar across all the 20th component when 800 training data were used, stations. JANUARY 2023 MO M O I E T A L. 7 FIG. 3. Relationship between the number of principal components and the RMSE for rainfall data. Solid and dotted lines are the results from rain-1D and rain-2D data, respectively. Black lines indicate the cumulative contribution ratio. As compared with PCA, the neural network approach can 2) SPATIAL DISTRIBUTION OF MAXIMUM INUNDATION DEPTH efficiently reduce the number of feature values through the use of an activation function (ReLU) and convolutional In the previous section, the dimension of the rainfall data was layers. As discussed above, we only conducted dimensionality reduced by solely focusing on the features of the temporal distri- reduction of the rain-1D data with CNN-AE. Figure 4 shows bution of rainfall data at individual AMeDAS stations (rain-1D the results of the PCA and CNN-AE approaches for the data). However, the feature extraction of the depth-MAP is rain-1D data. The RMSE values of VAL data obtained by the likelyto beoverfitted because it requires direct reduction of the CNN-AE were lower than the values obtained by PCA for dimension of the depth-MAP. Furthermore, there is no method the rain-1D data when the number of features was .20. In for increasing the amount of effective training data other than by other words, the nonlinear NN-based AE would not always increasing the amount of ensemble simulation data, although the outperform the PCA when the number of features is too number of variables is significantly more in the depth-MAP than small. Therefore, this result indicates the importance of com- in the rainfall data. Figure 5 shows the results of PCA and paring NN-based AE with PCA for identifying whether CNN-AE for the depth-MAP. Relative to the cumulative contri- NN-based AE is more suitable than PCA. Architecture optimi- bution of PCA for the rain-2D data, the contribution of PCA for zation for a low number of features may enable the CNN-AE the depth-MAP was high for an identical number of principal to extract more features than PCA when there is a low number of features (e.g., 10), but investigating this point is beyond the components. When 800 training data were used, 112 and 5 fea- scope of present study. tures were necessary to extract the principal components of the FIG. 4. Relationship between the number of principal components (features) and the RMSE values derived for CNN-AE and PCA for rainfall data. Solid lines and dots are the results derived by PCA and CNN-AE for rain-1D data, respectively. Black lines indicate the cumulative contribution ratio for PCA. 8 AR TI F I C I A L I N T E LLI G E N C E F O R T H E E AR TH S Y S T E M S VOLUME 2 FIG.5.As in Fig. 4, but for depth-MAP data. rain-2D and depth-MAP data within a 95% cumulative contribu- rain-1D data, the feature connecting layer from the rain-1D data to depth-MAP, and the reconstructing layer for the depth- tion, respectively. This finding suggests that the number of impor- MAP (Table 3). The numbers of features of the rain-1D and tant features in the depth-MAP is significantly lower; moreover, Depth-MAP were empirically determined to be 10 and 81 form the depth-MAP obtained from the physical-based RRI model the RMSE decay curve (Figs. 4 and 5). Section 3b(1) evaluates was nearly reconstructed with a minimum of 5 features. The Rain2Depth with 5FCV through comparison between the PCA RMSE of the PCA for the depth-MAP decreased as the number and CNN-AE during feature extraction for the rain-1D and of training data increased, similar to the results of PCA for the depth-MAP. Then, section 3b(2) provides the result of the appli- rain-1D data. The CNN-AE for the depth-MAP consists of the cation of the TEST data and compares the maximum inundation two-dimensional convolutional layers (before feature extraction depth at point A with K20. by the fully connected layer in the encoder), as well as the two- 1) EVALUATION WITH 5FCV dimensional transposed convolutional layers in the decoder (Table 2). Although there remains challenges in optimizing the This section evaluates Rain2Depth by comparing the dimen- CNN-AE architecture, the CNN-AE extracts features more ef- sionality reduction techniques of PCA and CNN-AE. Figure 6 fectively than does PCA when the number of neurons in the fully shows the RMSE of the depth-MAP predicted by Rain2Depth connected layer is ,20 with 800 training data. with PCA and CNN-AE. The RMSE of VAL predicted by Rain2Depth was larger than the RMSE predicted by PCA and b. Emulation of the physical-based RRI model with the CNN-AE, as demonstrated in Fig. 5. Thisisattributable tothe neural network Rain2Depth nonlinear relationship of features between the rain-1D data and Rain2Depth is an emulator of a physical-based RRI model the depth-MAP data. By increasing the number of training data, and consists of three parts: the feature extraction layer for the RMSE improved from 2.92 (200 data) to 2.18 cm (800 data) FIG. 6. Relationship between training data size and RMSE of depth-MAP emulated with dimensional reduction us- ing (a) PCA and (b) CNN-AE. Red, blue, and black dots show the results from TRAIN, VAL, and TEST data, re- spectively. Error bars show the standard deviation of 5FCV; The blue dotted line shows a power-law fit line from the RMSE of VAL data. JANUARY 2023 MO M O I E T A L. 9 FIG. 7. Example of TEST data for (a) the maximum inundation depth at point-A and (b) the total water volume in the target area (red-outlined box in Fig. 8a, below), obtained using Rain2Depth with CNN-AE, Rain2Depth with PCA, and K20. in Rain2Depth with PCA; it improved from 1.82 (200 data) to 2) PERFORMANCE WITH TEST DATA 1.38 cm (800 data) in Rain2Depth with CNN-AE. The larger re- This study aimed to reduce computational costs by emulating sidual errors related to nonlinearity could be reduced by using a the physical-based RRI model. Through 5FCV for several data very large amount of data to train Rain2Depth. Comparison of sizes, we determined that a very large amount of training data is dimensionality reduction techniques showed that the RMSE of needed to predict within 1 cm accuracy the maximum inunda- the CNN-AE was smaller than the RMSE of the PCA because tion depth when .1000 data are used. In this section, we de- the CNN-AE technique can extract more information than the scribe the performance of Rain2Depth trained by all TRAIN/ PCA technique with the same number of features. When the VAL data (total of 1000 data) and apply it to the TEST data RMSE meets the power law, usingatrainingdatanumber N of 20.2 20.2 4 (500 data), which are independent from the Rain2Depth train- 8.3 3 N for PCA and 5.3 3 N for CNN-AE, ;4 3 10 ing data, as described in section 2d. We also conducted a com- training data for Rain2Depth with PCA and ;4 3 10 training parison of the maximum inundation depth at point A with the data for Rain2Depth with CNN-AE were required to attain an result of a previous study (K20), as described in section 2e. RMSE of 1 cm in VAL, similar to the RMSE of TRAIN. For an The RMSE for the 1000 training data is shown in Fig. 6. RMSE of 1.4 cm in VAL, which is identical to the RMSE for The RMSE of TEST using Rain2Depth with PCA and Rain2Depth with CNN-AE using 800 training data, the Rain2- CNN-AE meets the power law fitted according to the results of DepthwithPCA required ;7.5 3 10 training data. Therefore, our approach (i.e., Rain2Depth with CNN-AE) can train a net- 5FCV, which combines 1000 ensemble data for TRAIN and work more efficiently, relative to the other emulators described VAL, independent of TEST. This result indicates that the eval- in this paper. uation using VAL data is meaningful. Figure 7a shows the 10 AR TI F I C I A L I N T E LLI G E N C E F O R T H E E AR TH S Y S T E M S VOLUME 2 FIG. 8. Example of the depth-MAP obtained using (a) the physics-based RRI model, (b) Rain2Depth with CNN-AE, and (c) Rain2Depth with PCA. Also shown is the spatial distribution of the difference between the physics-based RRI model and (d) Rain2Depth with CNN-AE or (e) Rain2Depth with PCA. The red-outlined box is the target area, which has frequent inundation. performance of the maximum inundation depth at point A potential to predict the spatial distribution of the maximum in- emulated by Rain2Depth with PCA and CNN-AE, as well as undation depth better than Rain2Depth with PCA. K20, for TEST data. The RMSE of the maximum inundation depth at point-A is larger than the result in Fig. 6. This is because 4. Conclusions Fig. 6 includes the noninundated pixels in all the events, whereas In this study, we developed an emulator, Rain2Depth, of the point-A is an extremely inundated area. This indicates that the physical-based RRI model with a deep convolutional neural reconstructing in inundated areas has a difficultybyan emulator network. This network consists of feature extraction of the trained with the small data. The maximum inundation depth ob- rainfall data, feature transposition from the rainfall to the spa- tained using Rain2Depth is better than the result obtained using tial distribution of the maximum inundation depth, and recon- K20, possibly because of nonlinear transformation by the middle struction of the maximum inundation depth using spatial network in Rain2Depth (Table 3). For the total water volume in distribution features. To extract the rainfall and inundation fea- the target area, which has frequent inundation (Fig. 8), Rain2- tures, we used two approaches: PCA and CNN-AE. Because Depth with CNN-AE has the better agreement (Fig. 7b)witha CNN-AE can extract nonlinear features, the RMSE of the re- physical-based model than PCA because CNN-AE trained the constructed data with CNN-AE was smaller than the RMSE relationship with neighbor pixels by convolutional layers. with PCA. Thus, the dimensionality reduction by CNN-AE Figure 8 shows the example of the depth-MAP of TEST data was suitable for constructing an emulator of the physical-based obtained using Rain2Depth with PCA and CNN-AE. There is inundation around the Omono River, especially at serpentine RRI model while maintaining a small network trained with a and downstream locations (Fig. 8a). These inundation charac- small number of training data. teristics were retrieved by Rain2Depth with both PCA and Rain2Depth was constructed using a deep neural network to CNN-AE. The result shows Rain2Depth with CNN-AE had a connect the features of input and output variables. The results JANUARY 2023 MO M O I E T A L. 11 according to 5FCV indicated that Rain2Depth with CNN-AE is REFERENCES better than Rain2Depth with PCA when using the same num- Abe, S., S. Watanabe, M. Yamada, S. Kotsuki, and A. Watanuki, ber of features. In comparison with the previous study (K20), in 2019: Effects of precipitation observations and sea surface terms of maximum inundation depth at a specific location, we temperature patterns on inundation analysis using large-scale showed that Rain2Depth with CNN-AE was the best emulator climate prediction information (in Japanese with English among the three models because the middle layer in Rain2Depth abstract). J. Japan Soc. Civil Eng., 75,I_1081–I_1086, https:// provides nonlinear transformation. Thus, by using the middle doi.org/10.2208/jscejhe.75.2_I_1081. network in Rain2Depth (instead of K20) to emulate the maxi- Ba, J. L., J. R. Kiros, and G. E. Hinton, 2016: Layer normaliza- mum inundation depth at point A, we achieved better perfor- tion. arXiv, 1607.06450v1, https://arxiv.org/abs/1607.06450. Bocquet, M., A. Farchi, and Q. Malartic, 2021: Online learning of mance than K20 but worse performance than Rain2Depth both state and dynamics using ensemble Kalman filters. Found. with CNN-AE. Unlike K20, Rain2Depth can predict the spa- Data Sci., 3, 305–330, https://doi.org/10.3934/fods.2020015. tial distribution of maximum inundation depth. Chang, L.-C., H.-Y. Shen, Y.-F. Wang, J.-Y. Huang, and Y.-T. Lin, For future work, we have the three goals: evaluation of the 2010: Clustering-based hybrid inundation model for forecast- emulator through application to the actual inundation event ing flood inundation depths. J. Hydrol., 385,257–268, https:// (e.g., July 2017), optimization of the network with a small num- doi.org/10.1016/j.jhydrol.2010.02.028. ber of training data, and application to more complicated physics Gro ¨nquist, P., C. Yao, T. Ben-Nun, N. Dryden, P. Dueben, S. Li, processes (e.g., time series of the spatial distribution of inunda- and T. Hoefler, 2021: Deep learning for post-processing tion depth). Such improvements may enable the establishment of ensemble weather forecasts. Philos. Trans. Roy. Soc., A379, an operational flood warning system. 20200092, https://doi.org/10.1098/rsta.2020.0092. Japan suffers from an increasing trend of floods and inunda- Hashimoto, M., and T. Nakajima, 2017: Development of a remote tions induced by severe precipitation. Improving real-time sensing algorithm to retrieve atmospheric aerosol proper- inundation predictions is important to mitigate such disasters ties using multiwavelength and multipixel information. J. Geophys. Res. Atmos., 122,6347–6378, https://doi.org/10. from reservoir operations and provide sufficient time for resi- 1002/2016JD025698. dents’ evacuation. For that purpose, operational real-time sys- Hess, P., and N. Boers, 2022: Deep learning for improving numerical tems have been installed for major rivers in Japan using physical weather prediction of heavy rainfall. J. Adv. Model. Earth Syst., rainfall–runoff-inundation models with assimilation of water 14, e2021MS002765, https://doi.org/10.1029/2021MS002765. depth observation. There can be two possible applications to Hochreiter, S., and J. Schmidhuber, 1997: Long short-term memory. bring our methods to operational systems. Neural Comput., 9, 1735–1780, https://doi.org/10.1162/neco.1997. The first application is the use for computationally inexpensive 9.8.1735. inundation predictions. The second application is the use for the Ioffe, S., and C. Szegedy, 2015: Batch normalization: Accelerating data assimilation system of the RRI models. Because of the non- deep network training by reducing internal covariate shift. linearity of rainfall–runoff, river, and inundation processes, parti- arXiv, 1502.03167v3, https://arxiv.org/abs/1502.03167v3. cle filters would be a more suitable data assimilation method for Jhong, B.-C., J.-H. Wang, and G.-F. Lin, 2017: An integrated two- RRI models than ensemble Kalman filters and variational meth- stage support vector machine approach to forecast inundation maps during typhoons. J. Hydrol., 547, 236–252, https://doi.org/ ods. However, the particle filter requires many more ensembles 10.1016/j.jhydrol.2017.01.057. with more computations for employing RRI simulations. Here Keisler, R., 2022: Forecasting global weather with graph neural emulators could be used for increasing ensembles as augmented networks. arXiv, 2202.07575v1, https://doi.org/10.48550/arXiv. members (Kotsuki and Bishop 2022). In addition to such meth- 2202.07575. odological developments, tight discussions with operational Kikuchi, R., T. Misaka, and S. Obayashi, 2016: International journal administrations are also necessary. Such further updates toward of computational fluid dynamics real-time prediction of operational systems are important directions of our future studies. unsteady flow based on POD reduced-order model and particle filter. Int. J. Comput. Fluid Dyn., 30, 285–306, https://doi.org/10. Acknowledgments. This study was partly supported by the 1080/10618562.2016.1198782. Japan Society for the Promotion of Science (JSPS) KAKENHI Kiranyaz, S., R. Ince, R. Hamila, and M. Gabbouj, 2015: Convolu- Grants JP21J01854, JP21H04571, and 22H05230, JST PRESTO tional neural networks for patient-specificECG classification. 37th Annual Int. Conf. IEEE Engineering in Medicine and MJPR1924, Ministry of Education, Culture, Sports, Science and Biology Society (EMBC), Milan, Italy, IEEE, 2608–2611, Technology of Japan (JPMXP1020200305) under the “Program https://doi.org/10.1109/EMBC.2015.7318926. for Promoting Researches on the Supercomputer Fugaku” }}, O. Avci, O. Abdeljaber, T. Ince, M. Gabbouj, and D. J. (Large Ensemble Atmospheric and Environmental Prediction Inman, 2021: 1D convolutional neural networks and appli- for Disaster Prevention and Mitigation), the Ministry of Land, cations: A survey. Mech. Syst. Signal Process., 151, 107398, Infrastructure, Transportation and Tourism of Japan, and the https://doi.org/10.1016/j.ymssp.2020.107398. IAAR Research Support Program of Chiba University. Kobayashi, K., L. Duc, Apip, T. Oizumi, and K. Saito, 2020: Ensemble flood simulation for a small dam catchment in Data availability statement. Because of the large volume of Japan using nonhydrostatic model rainfalls}Part 2: Flood involved data and limited disk space available, the data will forecasting using 1600-member 4D-EnVar-predicted rain- be shared online upon request to author Masahiro Momoi falls. Nat. Hazards Earth Syst. Sci., 20,755–770, https://doi. (momoi-masahiro@doerresearch.com). org/10.5194/nhess-20-755-2020. 12 AR TI F I C I A L I N T E LLI G E N C E F O R T H E E AR TH S Y S T E M S VOLUME 2 Konja, A., Y. Nakamura, S. Abe, T. Sayama, and Y. Wakazuki, 2018: Pathak, J., and Coauthors, 2022: Fourcastnet: A global data-driven high-resolution weather model using adaptive Fourier neural Analysis of the flood control effect of the Kinugawa upper operators. arXiv, 2202.11214v1, https://doi.org/10.48550/arXiv. stream dam during the September 2015 Kanto-Tohoku heavy 2202.11214. rain (in Japanese with English abstract). J. Hydraul. Eng., 62, Ravuri, S., and Coauthors, 2021: Skilful precipitation nowcasting I_1507–I_1512, https://doi.org/10.2208/jscejhe.74.I_1507. using deep generative models of radar. Nature, 597, 672–677, Kotsuki, S., M. Momoi, R. Kikuchi, S. Watanabe, M. Yamada, https://doi.org/10.1038/s41586-021-03854-z. S. Abe, and A. Watanuki, 2020: Emulating rainfall-runoff- Sayama, T., G. Ozawa, T. Kawakami, S. Nabesaka, and K. Fukami, inundation model through ensemble learning of multiple 2012: Rainfall–runoff–inundation analysis of the 2010 Pakistan regularized regressors (in Japanese with English abstract). flood in the Kabul River basin. Hydrol. Sci. J., 57, 298–312, Ann. J. Hydraul. Eng., 76, 367–372, https://doi.org/10.2208/ https://doi.org/10.1080/02626667.2011.644245. jscejhe.76.2_I_367. Shi, C., M. Hashimoto, K. Shiomi, and T. Nakajima, 2020: Devel- }}, and C. H. Bishop, 2022: Implementing hybrid background opment of an algorithm to retrieve aerosol optical properties error covariance into the LETKF with attenuation-based over water using an artificial neural network radiative trans- localization: Experiments with a simplified AGCM. Mon. fer scheme: First result from GOSAT-2/CAI-2. IEEE Trans. Wea. Rev., 150, 283–302, https://doi.org/10.1175/MWR-D- Geosci. Remote Sens., 59, 9861–9872, https://doi.org/10.1109/ 21-0174.1. TGRS.2020.3038892. Krizhevsky, A., I. Sutskever, and G. E. Hinton, 2012: ImageNet Shi, X., Z. Gao, L. Lausen, H. Wang, D.-Y. Yeung, W.-K. Wong, classification with deep convolutional neural networks. Proc. and W.-C. Woo, 2017: Deep learning for precipitation nowcast- 25th Int. Conf. Neural Information Processing Systems,Vol. ing: A benchmark and a new model. Proc. 31st Int. Conf. on 1, Lake Tahoe, NV, Association for Computing Machinery, Neural Information Processing Systems, Vol. 30, Long Beach, 1097–1105, https://dl.acm.org/doi/10.5555/2999134.2999257. CA, Association for Computing Machinery, 5622–5632, https:// Liang, J., K. Terasaki, and T. Miyoshi, 2021: A machine learning dl.acm.org/doi/10.5555/3295222.3295313. approach to the observation operator for satellite radiance Takenaka, H., T. Y. Nakajima, A. Higurashi, A. Higuchi, T. data assimilation. 23rd EGU General Assembly,New Orleans, Takamura, R. T. Pinker, and T. Nakajima, 2011: Estimation LA, Amer. Geophys. Union, Abstract EGU21-10475, https:// of solar radiation using a neural network based on radiative doi.org/10.5194/egusphere-egu21-10475. transfer. J. Geophys. Res., 116, D08215, https://doi.org/10.1029/ Lin, G.-F., H.-Y. Lin, and Y.-C. Chou, 2013: Development of a 2009JD013337. real-time regional-inundation forecasting model for the inunda- Tsuyuki, T., and R. Tamura, 2022: Nonlinear data assimilation tion warning system. J. Hydroinf., 15, 1391–1407, https://doi.org/ by deep learning embedded in an ensemble Kalman filter. 10.2166/hydro.2013.202. J. Meteor. Soc. Japan, 100,533–553, https://doi.org/10. Lumley, J. L., 1967: The structure of inhomogeneous turbulent 2151/jmsj.2022-027. flows. Atmospheric Turbulence and Radio Wave Propagation, Van, S. P.,H.M. Le, D. V. Thanh, T. D. Dang,H.H.Loc,and A. M. Yaglom, Ed., Nauka, 166–178. D. T. Anh, 2020: Deep learning convolutional neural network Makinoshima, F., Y. Oishi, T. Yamazaki, T. Furumura, and in rainfall–runoff modelling. J. Hydroinf., 22, 541–561, https:// F. Imamura, 2021: Early forecasting of tsunami inundation doi.org/10.2166/hydro.2020.095. from tsunami and geodetic observation data with convolu- Watanabe, S., M. Yamada, S. Abe, and M. Hatono, 2020: Bias cor- tional neural networks. Nat. Commun., 12, 2253, https://doi. rection of d4PDF using a moving window method and their org/10.1038/s41467-021-22348-0. uncertainty analysis in estimation and projection of design rain- Mizuta, R., and Coauthors, 2017: Over 5,000 years of ensemble fall depth. Hydrol. Res. Lett., 14, 117–122, https://doi.org/10. future climate simulations by 60-km global and 20-km regional 3178/hrl.14.117. atmospheric models. Bull. Amer. Meteor. Soc., 98, 1383–1398, Wu, B., T. Zhou, and T. Li, 2009: Seasonally evolving dominant in- https://doi.org/10.1175/BAMS-D-16-0099.1. terannual variability modes of East Asian climate. J. Climate, Mosavi, A., P. Ozturk, and K.-W. Chau, 2018: Flood prediction 22, 2992–3005, https://doi.org/10.1175/2008JCLI2710.1. using machine learning models: Literature review. Water, 10, Yamazaki, D., S. Kanae, H. Kim, and T. Oki, 2011: A physically 1536, https://doi.org/10.3390/w10111536. based description of floodplain inundation dynamics in a Murray, N. E., and L. S. Ukeiley, 2007: An application of Gappy global river routing model. Water Resour. Res., 47, W04501, POD. Exp. Fluids, 42,79–91, https://doi.org/10.1007/s00348- https://doi.org/10.1029/2010WR009726. 006-0221-y. Yashiro, H., and Coauthors, 2020: A 1024-member ensemble data Nair, V., and G. E. Hinton, 2010: Rectified linear units improve assimilation with 3.5-km mesh global weather simulations. restricted Boltzmann machines. Proc. Int. Conf. Machine SC20: Int. Conf. for High Performance Computing, Network- Learning, Haifa, Israel, Association for Computing Machin- ing, Storage and Analysis, Atlanta, GA, IEEE, 1–10, https:// ery, 807–814, https://dl.acm.org/doi/10.5555/3104322.3104425. doi.org/10.1109/SC41405.2020.00005. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Artificial Intelligence for the Earth Systems American Meteorological Society

Emulating Rainfall–Runoff-Inundation Model Using Deep Neural Network with Dimensionality Reduction

Download PDF
12 pages

Loading next page...
 
/lp/american-meteorological-society/emulating-rainfall-runoff-inundation-model-using-deep-neural-network-QIzVoGVdCU

References (1)

Copyright
Copyright © American Meteorological Society
eISSN
2769-7525
DOI
10.1175/aies-d-22-0036.1
Publisher site
See Article on Publisher Site

Abstract

JANUARY 2023 MO M O I E T A L. 1 Emulating Rainfall–Runoff-Inundation Model Using Deep Neural Network with Dimensionality Reduction a,b c,d,e,f g,b g h MASAHIRO MOMOI , SHUNJI KOTSUKI, RYOTA KIKUCHI, SATOSHI WATANABE, MASAFUMI YAMADA, AND SHIORI ABE GRASP SAS, Lezennes, France DoerResearch, Inc., Nagoya, Japan Institute for Advanced Academic Research, Chiba University, Chiba, Japan Center for Environmental Remote Sensing, Chiba University, Chiba, Japan RIKEN Center for Computational Science, Kobe, Japan RPRESTO, Japan Science and Technology Agency, Chiba, Japan Office of Society Academia Collaboration for Innovation, Kyoto University, Kyoto, Japan Disaster Prevention Research Institute, Kyoto University, Kyoto, Japan Mitsui Consultants Co., Ltd., Tokyo, Japan (Manuscript received 23 May 2022, in final form 14 November 2022) ABSTRACT: Predicting the spatial distribution of maximum inundation depth (depth-MAP) is important for the mitigation of hydrological disasters induced by extreme precipitation. However, physics-based rainfall–runoff-inundation (RRI) models, which are used operationally to predict hydrological disasters in Japan, require massive computational resources for numerical simulations. Here, we aimed at developing a computationally inexpensive deep learning model (Rain2Depth) that emulates an RRI model. Our study focused on the Omono River (Akita Prefecture, Japan) and predicted the depth-MAP from spatial and temporal rainfall data for individual events. Rain2Depth was developed based on a convolutional neural network (CNN) and predicts depth-MAP from 7-day successive hourly rainfall at 13 rain gauge stations in the basin. For training the Rain2Depth, we simulated the depth-MAP by the RRI model forced by 50 ensembles of 30-yr data from large-ensemble weather/climate predictions. Instead of using the input and output data directly, we extracted important features from input and output data with two dimensionality reduction techniques [principal component analysis (PCA) and the CNN approach] prior to training the network. This dimensionality reduction aimed to avoid overfitting caused by insufficient training data. The nonlinear CNN approach was superior to the linear PCA for extracting features. Finally, the Rain2Depth architecture was built by connecting the extracted features between input and output data through a neural network. Rain2Depth-based predictions were more ac- curate than predictions from our previous model (K20), which used ensemble learning of multiple regularized regressions for a specific station. Whereas the K20 can predict maximum inundation depth only at stations, our study achieved depth-MAP pre- diction by training only the single model Rain2Depth. KEYWORDS: Flood events; Runoff; Deep learning; Dimensionality reduction 1. Introduction The ensemble NWP forecasts enable probabilistic flood fore- casts in which ensemble NWP forecasts are used as input for Recent advances in high-performance computers (HPCs) have ensemble hydrological simulations. In the past two decades, enabled numerical weather prediction (NWP) at high spatial res- extensive hydrological studies have been advancing knowledge olution and with ensembles of .1000 members. For example, of the physical processes of river discharge, runoff, and two- Yashiro et al. (2020) successfully conducted a global atmospheric dimensional inundation (e.g., Sayama et al. 2012; Yamazaki data assimilation experiment at 3.5-km resolution with 1024 et al. 2011). Recent studies carry out probabilistic flood forecasts ensembles using the Fugaku, the flagship supercomputer of using ensemble NWP data (e.g., Kobayashi et al. 2020). Owing to Japan. In the past decade, the computational resources of HPCs the progress of physical-based models, realistic simulation can be have been increased by adding cores of the central processing conducted for an operational flood warning system. However, unit. For such many-core HPCs, increasing the number of ensem- the expanded physical-based models require progressively more ble members of NWP models is more scalable than reducing the and more computational resources, as observed in NWP models. horizontal and vertical resolutions because ensemble forecasts For real-time flood warning systems, the exploration of computa- can be essentially parallelized. Therefore, a Japanese project is tionally inexpensive inundation prediction methods is an impor- intended to prevent and mitigate weather-related disas- tant alternative to enhanced ensemble NWPs. For that purpose, ters through the effective use of large-ensemble weather this study uses deep neural network techniques. predictions. Recently, applications of deep learning in NWP have been massively investigated. One application is to use deep learning for data-driven weather prediction such as precipitation nowcast- ing (e.g., Shi et al. 2017; Ravuri et al. 2021). Also, using deep learning in the postprocessing step to adjust model outputs Corresponding author: Masahiro Momoi, momoi-masahiro@ doerresearch.com is also known to be beneficial (e.g., Gronquist et al. 2021; DOI: 10.1175/AIES-D-22-0036.1 e220036 Ó 2023 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses). 2 AR TI F I C I A L I N T E LLI G E N C E F O R T H E E AR TH S Y S T E M S VOLUME 2 Hess and Boers 2022). Combining deep learning and data flooded rivers in Japan; we compare the result with the physical- assimilation is also well-suited to applications such as learn- based model simulation. ing data assimilation (e.g., Bocquet et al. 2021; Tsuyuki and This study is organized as follows. Section 2 describes Tamura 2022) and learning observation operators (e.g., Liang the methods and experimental settings. Section 3 reveals et al. 2021). Challenging studies aim at emulating NWP models the experimental results and provides discussion. Section 4 for learning relations between input and output weather data provides a summary. (Pathak et al. 2022; Keisler 2022). Namely, these studies use deep learning for emulating meteorological predictions, model 2. Methods/experimental design output adjustments, data assimilation, observation operator, This study proposes a machine learning model, called and atmospheric dynamical processes. Rain2Depth, that computes the spatial distributions of max- The emulation of physical-based models using neural net- imum inundation depth from input rainfall data (Fig. 1). works has been also investigated broadly in the Earth sciences. Training inundation data for Rain2Depth were generated For example, in atmospheric radiation studies, progress to- by a physical-based rainfall–runoff-inundation (RRI) model ward addressing the lack of computational resources has been (Sayama et al. 2012), an operational real-time flood prediction made by moving from physical-based to machine learning– model used in Japan (e.g., in Hyogo Prefecture). The Policy based models. Takenaka et al. (2011) and Shi et al. (2020) re- Decision Making for Future Climate Change (d4PDF; Mizuta constructed the output (i.e., sky radiance and radiative flux) of et al. 2017) database was used for the input rainfall data, as the radiative transfer model; quasi-real-time processing of described in section 2a. The Rain2Depth model is based on a satellite observations was successfully achieved (e.g., neural network (section 2c). To enhance generality, we applied Hashimoto and Nakajima 2017). Inspired by these previous dimensionality reduction techniques to input and output data studies, we aim to develop a deep learning–based inundation prior to neural network training (section 2b). The models were prediction method, called Rain2Depth, that emulates the cali- evaluated with cross validation, as described in section 2d. brated physical-based inundation model. In the previous stud- ies, most of the emulators of the inundation model have been a. Training data developed for emulating the time series of the inundation 1) TARGET RIVER BASIN:OMONO RIVER BASIN depth using rainfall successively and can be classified into the following two types: emulating at a particular site (e.g., Mosavi The Omono River, with a basin area of 4710 km , is a class- et al. 2018) and in the area spatially (e.g., Chang et al. 2010; A river located in the northeastern part of Japan. It is one of Lin et al. 2013; Jhong et al. 2017). The study on the emulator the most frequently flooded rivers in Japan; it has been of a particular site has been investigated through more than flooded four times since 2000. Therefore, the development of 100 papers using various machine learning techniques in the rapid flood forecasting systems is critical in this region. Historical last 2 decades (Mosavi et al. 2018). floods have mainly been caused by the stationary baiu front. In particular, we aimed to develop a deep learning–based surrogate model of the state-of-the-art inundation model, es- 2) RAINFALL DATA OF D4PDF WITH BIAS CORRECTION pecially for evacuation plans in the early stages of rainfall. As Rainfall data obtained from d4PDF (Mizuta et al. 2017) compared with the abovementioned relevant studies, this were used as input for RRI simulations. The d4PDF provides study focuses on the maximum inundation depth during the the results of ensemble experiments that comprise more than event. Kotsuki et al. (2020) proposed a regression-based emu- 1000 years of meteorological data for both historical reproduc- lator of an inundation model and demonstrated good agree- tions and future projections. Specifically, the historical experi- ments with model-based predictions. However, the machine ments reproduced the past period of 1951–2010 with 20-km- proposed by Kotsuki et al. (2020) can only predict inundation resolution 50 ensembles, which was dynamically downscaled by depth at stations. Therefore, we proposed a new machine that 60-km global ensemble experiment driven by perturbed bound- can predict the spatial pattern of inundation depth using a ary conditions to sea surface and sea ice temperature. This study time series of distributed rainfall data. General deep neural used these 50 ensemble experiments, with a focus on the past networks, including networks in the aforementioned studies 30 years (1981–2010). We identified the event with the maxi- (Takenaka et al. 2011; Shi et al. 2020), require very large mum 30-h precipitation in each year. For the RRI simulations, amounts of training data to optimize massive parameters in we extracted successive 168-h (i.e., 7 day) precipitation data the network. In general, the high-resolution inundation simu- including the heaviest precipitation: 48-h spin up, 30-h heavy lation requires high-dimensional input data and produces precipitation, and 90-h rest periods. In total, 1500 events were high dimensional output data. However, for hydrological em- extracted for RRI simulation. ulators, there would be fewer essential features within input Prior to the RRI simulation, a bias correction method rainfall data and output inundation patterns. Here, we pro- pose training the network with features extracted by linear (Watanabe et al. 2020) was applied to the d4PDF rainfall data principal component analysis and the nonlinear neural for reducing unignorable bias in model-based precipitation network–based autoencoder (section 3a). We then build the data. The observation dataset of the Automated Meteorological Rain2Depth model by connecting input and output features, Data Acquisition System (AMeDAS) operated by the Japan as described in section 3b. In this study, we apply the proposed Meteorological Agency was used as reference data for the bias machine to the Omono River, one of the most frequently correction. This bias correction led to improved extreme JANUARY 2023 MO M O I E T A L. 3 FIG. 1. Conceptual design of this study; input data were the rainfall data generated from d4PDF with the bias correction method (Watanabe et al. 2020) described in section 2a(2); reference output data were the spatial distribution of maximum inundation depth generated from the rainfall data with the physical-based RRI model (Sayama et al. 2012) described in section 2a(3); K20 and Rain2Depth are the emula- tors developed by Kotsuki et al. (2020) and this study, respectively. Points A and B in the figure are the target sites for Kotsuki et al. (2020). precipitation events in the past. The operational design rainfall channels surveyed by the Ministry of Land, Infrastructure, Transportation and Tourism of Japan. In addition, we mod- of the Omono River, which is 258.7 mm per 2-day period, was reproduced with an error of ,10% using this bias correction eled the flood control operations of six dams in the Omono for the d4PDF. In this procedure, the rainfall data were discre- River (Konja et al. 2018). To model runoff processes, we ap- tized spatially into 13 AMeDAS stational data of 168 h. plied the unsaturated lateral flow mechanism for forest area and the saturated lateral flow mechanism for other land-use areas. The model was calibrated with two heavy rainfall events 3) RRI MODEL AND MAXIMUM INUNDATION DEPTH that occurred in 2004 and 2011; it was validated by focusing on The RRI model is a two-dimensional physical-based distrib- the record-breaking flood in 2017. The RRI model showed uted hydrological model, that simultaneously simulates both the good reproducibility in terms of the observed discharge, inun- rainfall–runoff process in slope areas and the flood inundation dation area and depth (Abe et al. 2019). For all six official dis- process in rivers and floodplains. In this section, we first describe charge observation points, Nash–Sutcliff efficiency values the RRI model setting followed by experimental setting to pro- were .0.70. The reproducibility of inundated and noninun- duce spatial inundation depth training data. dated areas was 90%. See Abe et al. (2019) for more details. This study used the RRI model calibrated and validated by Using the calibrated RRI model, we conducted experi- Abe et al. (2019) in the Omono River. The spatial resolution ments with the bias-corrected d4PDF rainfall data for 1500 of the model is 270 m. All grid cells with an upstream area of events. We used the maximum inundation depth of each grid .2km were regarded as river grid cells. For the river geometry to generate the spatial distribution of maximum inundation of main rivers, we included the cross-section shapes of river depth (depth-MAP) for each event. 4 AR TI F I C I A L I N T E LLI G E N C E F O R T H E E AR TH S Y S T E M S VOLUME 2 TABLE 1. Architecture of the convolutional autoencoder for rainfall data. Here and in subsequent tables, the numbers in brackets indicate the dimensions of the array in the program. Layer type Output channel Kernel size Padding Stride Encoder (Input shape: [13, 1, 168]) Conv1d 1 ReLU [13, 16, 168] 3 1 1 Conv1d 1 ReLU [13, 16, 168] 3 1 1 Conv1d 1 ReLU [13, 16, 168] 3 1 1 Conv1d 1 ReLU [13, 16, 168] 3 1 1 Conv1d 1 ReLU [13, 16, 84] 2 0 2 Conv1d 1 ReLU [13, 16, 84] 3 1 1 Conv1d 1 ReLU [13, 16, 84] 3 1 1 Conv1d 1 ReLU [13, 16, 28] 3 0 3 Conv1d 1 ReLU [13, 16, 14] 2 0 2 Linear 1 LayerNorm [13, 81] }} } Decoder (Input shape: [13, 81]) Linear [13, 224] }} } ConvTranspose1d 1 ReLU [13, 16, 28] 2 0 2 ConvTranspose1d 1 ReLU [13, 16, 84] 3 0 3 Conv1d 1 ReLU [13, 16, 84] 3 1 1 Conv1d 1 ReLU [13, 16, 84] 3 1 1 ConvTranspose1d 1 ReLU [13, 16, 168] 2 0 2 Conv1d 1 ReLU [13, 16, 168] 3 1 1 Conv1d 1 ReLU [13, 16, 168] 3 1 1 Conv1d 1 ReLU [13, 16, 168] 3 1 1 Conv1d [13, 16, 168] 3 1 1 b. Dimensionality reduction techniques The present study designed the AE as follows. The CNN-AE for input rainfall data was constructed by one-dimensional con- Deep neural network training processes require massive volution (Conv1d). Conv1d was developed for electrocardiogram training data because such processes involve large numbers of classification (Kiranyaz et al. 2015). Recent studies have used network parameters depending on the features of the input Conv1d for time series data in geophysics (e.g., Makinoshima and output data. However, this study endeavored to train the et al. 2021; Van et al. 2020) because of its low computational cost network with a moderately small number of events (1500) as (Kiranyaz et al. 2021). For rainfall–runoff emulation, Conv1d training data. Therefore, it was beneficial to extract important may bemoresuitablethan longshort-term memory(Hochreiter features from input (rainfall) and output (depth MAP) data, and Schmidhuber 1997) because Conv1d effectively extracts the then use these extracted features to train a network with dependencies (features) in short-term time series (Van et al. reduced network parameters. Here, we extracted important 2020). Therefore, the features of the rainfall data were extracted features by reducing the data dimensionality. using the Conv1d, fully connected layer (“Linear” in the tables), The easiest way to reduce data dimensionality is principal com- rectified linear units (ReLU; Nair and Hinton 2010), and layer ponent analysis (PCA); this method reduces the dataset features normalization (“LayerNorm” in the tables; Ba et al. 2016), as after orthogonalization by singular value decomposition. PCA is shown in Table 1. The CNN-AE for the rainfall data was con- known as statistical recognition tools as empirical orthogonal structed using the one-dimensional convolutional layers and a functions (Wu et al. 2009) or proper orthogonal decomposition fully connected layer in the encoder, while one-dimensional (Lumley 1967) in meteorology and geophysical fluid dynamics transposed convolutional layers and a fully connected layer fields, for example, postprocessing tools (Murray and Ukeiley were used in the decoder. Layer normalization can constrain 2007) and dominant components analysis (Kikuchi et al. 2016). the network parameters by specific layer data without batch However, PCA would be suboptimal choice if major modes are data (Ba et al. 2016). Therefore, it is applicable for data with nonlinear. An alternative method is a neural network–based both large and small batch sizes. Layer normalization is also technique known as autoencoder (AE), which can extract fea- suitable for large variance data, such as the extreme weather tures through nonlinear multilayer neural networks with activa- data used in this study; other normalization techniques (e.g., tion functions. In this study, we used the convolutional–neural batch normalization; Ioffe and Christian 2015) are not suitable network (CNN) AE (CNN-AE). It uses CNN networks as for large variance data. encoder and decoder before and after feature extraction using a Two-dimensional convolutional neural networks for image fully connected layer. Because the networks (e.g., activation data are rapidly progressing technologies (e.g., Krizhevsky function, normalization, and layer numbers) were built empiri- et al. 2012). A two-dimensional convolutional layer (Conv2d) cally using training and validation (TRAIN/VAL) data, there convolves the neighbor pixels of a target pixel and, thus, poten- might still be room for optimization. This study compared the tially extracts spatially distributed local features. These character- efficiency of data reduction between PCA and AE methods. istics would be beneficial for the feature extraction of depth-MAP JANUARY 2023 MO M O I E T A L. 5 TABLE 2. Architecture of the convolutional autoencoder for depth-MAP. Layer type Output channel Kernel size Padding Stride Encoder (Input shape: [1, 1, 432, 324]) Conv2d 1 ReLU [1, 128, 216, 162] 2 0 2 Conv2d 1 ReLU [1, 128, 108, 81] 2 0 2 Conv2d 1 ReLU [1, 128, 36, 27] 3 0 3 Conv2d 1 ReLU [1, 128, 12, 9] 3 0 3 Conv2d 1 ReLU [1, 128, 4, 3] 3 0 3 Linear 1 LayerNorm [1, 10] }} } Decoder (Input shape: [1, 10]) Linear [1, 1536] }} } ConvTranspose2d 1 ReLU [1, 128, 12, 9] 3 0 3 ConvTranspose2d 1 ReLU [1, 128, 36, 27] 3 0 3 ConvTranspose2d 1 ReLU [1, 128, 108, 81] 3 0 3 ConvTranspose2d 1 ReLU [1, 128, 216, 162] 2 0 2 ConvTranspose2d [1, 1, 432, 324] 2 0 2 because the inundated area should be continuously distributed trials. For the validation of the small set of training data, the around the river. In this study, we constructed the CNN-AE for score of the network was determined from the evaluation depth-MAP data by using a two-dimensional convolutional score of each trial derived from subtrials (Fig. 2b). For example, neural network (Table 2). when 400 data were used for training, the score of each trial was derived from 6 [i.e., combination C(4, 2)] subtrials. c. Neural network architectures e. Brief outline of an emulator with multiple To emulate the maximum inundation depth from rainfall data regularized regressors (i.e., Rain2Depth), we constructed a neural network by connect- ing the features of rainfall and the depth-MAP data extracted by Kotsuki et al. (2020) emulated the maximum inundation dimensionality reduction techniques, as described in section 2b. depth at the two locations denoted as points A and B in Fig. 1 The network consisted of the one-dimensional convolutional net- through ensemble learning with multiple regularized regres- work, ReLU, and the fully connected layer, as shown in Table 3. sors (herein this method is referred to as K20). The architec- ture of their emulator is illustrated in Fig. 1. It consists of the d. Cross validation following two aspects: maximum inundation depth at the spe- The dataset was grouped into 1000 data for training (labeled cific location predicted by three regressors with different regu- as TRAIN)/validation (labeled as VAL) and 500 data for tests larization [ridge, least absolute shrinkage and selection (labeled as TEST), as shown in Fig. 2.TEST data were not operator (LASSO), and elastic net] and ensemble learning us- used for emulator validation and architecting, as detailed in ing a random-forest classifier. In this study, K20 was trained sections 3a and 3b(1). After the network had been architected using TRAIN/VAL data (1000 data) as described by Kotsuki [section 3b(2)], the pseudo generalization performance of the et al. (2020) with the hyperparameters of regularization deter- network was estimated from TEST data that consist of inde- mined in their paper. pendent data from the TRAIN/VAL training procedure. Vali- dations of the models were performed via fivefold cross 3. Results and discussion validation (5FCV) using TRAIN/VAL and a small training da- taset that allowed assessment of training model feasibility. The a. Dimensionality reduction 5FCV divides the TRAIN/VAL into five subsets; it uses four 1) RAINFALL DATA subsets (total of 800 data) for TRAIN and the remaining sub- set (200 data) for VAL (Fig. 2a). Therefore, the evaluation Because the rainfall data of 13 AMeDAS stations con- score of the network was determined from the mean of five tained hourly data over seven days, the rainfall data had two TABLE 3. Architecture of the convolutional neural network of the middle layer in Rain2Depth. Layer type Output channel Kernel size Padding Stride (Input shape: [1, 81, 13]) Conv1d 1 ReLU [1, 200, 13] 1 0 1 Conv1d 1 ReLU [1, 200, 13] 1 0 1 Conv1d 1 ReLU [1, 200, 13] 1 0 1 Conv1d 1 ReLU [1, 200, 13] 1 0 1 Conv1d 1 ReLU [1, 20, 1] 13 0 1 Linear [1, 10] }} } 6 AR TI F I C I A L I N T E LLI G E N C E F O R T H E E AR TH S Y S T E M S VOLUME 2 FIG.2.Concept of fivefold cross validation: (a) general description of 5FCV and (b) description for small numbers of data points. dimensions and 2184 variables (168 h 3 13 AMeDAS stations) suggesting that the dimension of the rain-2D data is signifi- in each event (herein, rain-2D data). This number exceeded cantly increased beyond 800 data. Therefore, additional train- the number of training data; thus, we reduced dimensionality ing data for dimensionality reduction are needed to retain by using PCA. The dashed lines in Fig. 3 show the PCA results generalization ability. This feature arises from two factors: the for 200–800 training data. The root-mean-squared errors temporal distribution of rainfall (i.e., time series of rainfall) (RMSEs) of TRAIN and VAL decreased as the number of and spatial distribution of rainfall of the 13 AMeDAS stations training data increased; the use of 800 data for training yielded (e.g., change in rainfall region position from south to north or the best performance. An increased number of principal com- another movement). If the features of the temporal distribu- ponents was expected to improve the performance of TRAIN tion of rainfall are identical at all AMeDAS stations, the rainfall data reconstruction, but the TRAIN rainfall data did dimension of the rainfall data observed at each AMeDAS not match the VAL rainfall data because of overfitting. When station (herein, rain1d) can be reduced with the same machine. 200 training data were used, the VAL performance was not This assumption also produces 13-fold more effective training significantly improved with respect to TRAIN performance data relative to rain-2D data. The dimension of the rain-1D when the number of principal components was .10. These data can be reduced while retaining generalization perfor- findings suggest that principal components beyond the 10th mance even when using 200 training data (solid lines in Fig. 3). component do not enhance generalization performance. These Thus, the features of the temporal distribution of the rainfall findings were also observed for principal components beyond (such as rain and stop mechanism) are similar across all the 20th component when 800 training data were used, stations. JANUARY 2023 MO M O I E T A L. 7 FIG. 3. Relationship between the number of principal components and the RMSE for rainfall data. Solid and dotted lines are the results from rain-1D and rain-2D data, respectively. Black lines indicate the cumulative contribution ratio. As compared with PCA, the neural network approach can 2) SPATIAL DISTRIBUTION OF MAXIMUM INUNDATION DEPTH efficiently reduce the number of feature values through the use of an activation function (ReLU) and convolutional In the previous section, the dimension of the rainfall data was layers. As discussed above, we only conducted dimensionality reduced by solely focusing on the features of the temporal distri- reduction of the rain-1D data with CNN-AE. Figure 4 shows bution of rainfall data at individual AMeDAS stations (rain-1D the results of the PCA and CNN-AE approaches for the data). However, the feature extraction of the depth-MAP is rain-1D data. The RMSE values of VAL data obtained by the likelyto beoverfitted because it requires direct reduction of the CNN-AE were lower than the values obtained by PCA for dimension of the depth-MAP. Furthermore, there is no method the rain-1D data when the number of features was .20. In for increasing the amount of effective training data other than by other words, the nonlinear NN-based AE would not always increasing the amount of ensemble simulation data, although the outperform the PCA when the number of features is too number of variables is significantly more in the depth-MAP than small. Therefore, this result indicates the importance of com- in the rainfall data. Figure 5 shows the results of PCA and paring NN-based AE with PCA for identifying whether CNN-AE for the depth-MAP. Relative to the cumulative contri- NN-based AE is more suitable than PCA. Architecture optimi- bution of PCA for the rain-2D data, the contribution of PCA for zation for a low number of features may enable the CNN-AE the depth-MAP was high for an identical number of principal to extract more features than PCA when there is a low number of features (e.g., 10), but investigating this point is beyond the components. When 800 training data were used, 112 and 5 fea- scope of present study. tures were necessary to extract the principal components of the FIG. 4. Relationship between the number of principal components (features) and the RMSE values derived for CNN-AE and PCA for rainfall data. Solid lines and dots are the results derived by PCA and CNN-AE for rain-1D data, respectively. Black lines indicate the cumulative contribution ratio for PCA. 8 AR TI F I C I A L I N T E LLI G E N C E F O R T H E E AR TH S Y S T E M S VOLUME 2 FIG.5.As in Fig. 4, but for depth-MAP data. rain-2D and depth-MAP data within a 95% cumulative contribu- rain-1D data, the feature connecting layer from the rain-1D data to depth-MAP, and the reconstructing layer for the depth- tion, respectively. This finding suggests that the number of impor- MAP (Table 3). The numbers of features of the rain-1D and tant features in the depth-MAP is significantly lower; moreover, Depth-MAP were empirically determined to be 10 and 81 form the depth-MAP obtained from the physical-based RRI model the RMSE decay curve (Figs. 4 and 5). Section 3b(1) evaluates was nearly reconstructed with a minimum of 5 features. The Rain2Depth with 5FCV through comparison between the PCA RMSE of the PCA for the depth-MAP decreased as the number and CNN-AE during feature extraction for the rain-1D and of training data increased, similar to the results of PCA for the depth-MAP. Then, section 3b(2) provides the result of the appli- rain-1D data. The CNN-AE for the depth-MAP consists of the cation of the TEST data and compares the maximum inundation two-dimensional convolutional layers (before feature extraction depth at point A with K20. by the fully connected layer in the encoder), as well as the two- 1) EVALUATION WITH 5FCV dimensional transposed convolutional layers in the decoder (Table 2). Although there remains challenges in optimizing the This section evaluates Rain2Depth by comparing the dimen- CNN-AE architecture, the CNN-AE extracts features more ef- sionality reduction techniques of PCA and CNN-AE. Figure 6 fectively than does PCA when the number of neurons in the fully shows the RMSE of the depth-MAP predicted by Rain2Depth connected layer is ,20 with 800 training data. with PCA and CNN-AE. The RMSE of VAL predicted by Rain2Depth was larger than the RMSE predicted by PCA and b. Emulation of the physical-based RRI model with the CNN-AE, as demonstrated in Fig. 5. Thisisattributable tothe neural network Rain2Depth nonlinear relationship of features between the rain-1D data and Rain2Depth is an emulator of a physical-based RRI model the depth-MAP data. By increasing the number of training data, and consists of three parts: the feature extraction layer for the RMSE improved from 2.92 (200 data) to 2.18 cm (800 data) FIG. 6. Relationship between training data size and RMSE of depth-MAP emulated with dimensional reduction us- ing (a) PCA and (b) CNN-AE. Red, blue, and black dots show the results from TRAIN, VAL, and TEST data, re- spectively. Error bars show the standard deviation of 5FCV; The blue dotted line shows a power-law fit line from the RMSE of VAL data. JANUARY 2023 MO M O I E T A L. 9 FIG. 7. Example of TEST data for (a) the maximum inundation depth at point-A and (b) the total water volume in the target area (red-outlined box in Fig. 8a, below), obtained using Rain2Depth with CNN-AE, Rain2Depth with PCA, and K20. in Rain2Depth with PCA; it improved from 1.82 (200 data) to 2) PERFORMANCE WITH TEST DATA 1.38 cm (800 data) in Rain2Depth with CNN-AE. The larger re- This study aimed to reduce computational costs by emulating sidual errors related to nonlinearity could be reduced by using a the physical-based RRI model. Through 5FCV for several data very large amount of data to train Rain2Depth. Comparison of sizes, we determined that a very large amount of training data is dimensionality reduction techniques showed that the RMSE of needed to predict within 1 cm accuracy the maximum inunda- the CNN-AE was smaller than the RMSE of the PCA because tion depth when .1000 data are used. In this section, we de- the CNN-AE technique can extract more information than the scribe the performance of Rain2Depth trained by all TRAIN/ PCA technique with the same number of features. When the VAL data (total of 1000 data) and apply it to the TEST data RMSE meets the power law, usingatrainingdatanumber N of 20.2 20.2 4 (500 data), which are independent from the Rain2Depth train- 8.3 3 N for PCA and 5.3 3 N for CNN-AE, ;4 3 10 ing data, as described in section 2d. We also conducted a com- training data for Rain2Depth with PCA and ;4 3 10 training parison of the maximum inundation depth at point A with the data for Rain2Depth with CNN-AE were required to attain an result of a previous study (K20), as described in section 2e. RMSE of 1 cm in VAL, similar to the RMSE of TRAIN. For an The RMSE for the 1000 training data is shown in Fig. 6. RMSE of 1.4 cm in VAL, which is identical to the RMSE for The RMSE of TEST using Rain2Depth with PCA and Rain2Depth with CNN-AE using 800 training data, the Rain2- CNN-AE meets the power law fitted according to the results of DepthwithPCA required ;7.5 3 10 training data. Therefore, our approach (i.e., Rain2Depth with CNN-AE) can train a net- 5FCV, which combines 1000 ensemble data for TRAIN and work more efficiently, relative to the other emulators described VAL, independent of TEST. This result indicates that the eval- in this paper. uation using VAL data is meaningful. Figure 7a shows the 10 AR TI F I C I A L I N T E LLI G E N C E F O R T H E E AR TH S Y S T E M S VOLUME 2 FIG. 8. Example of the depth-MAP obtained using (a) the physics-based RRI model, (b) Rain2Depth with CNN-AE, and (c) Rain2Depth with PCA. Also shown is the spatial distribution of the difference between the physics-based RRI model and (d) Rain2Depth with CNN-AE or (e) Rain2Depth with PCA. The red-outlined box is the target area, which has frequent inundation. performance of the maximum inundation depth at point A potential to predict the spatial distribution of the maximum in- emulated by Rain2Depth with PCA and CNN-AE, as well as undation depth better than Rain2Depth with PCA. K20, for TEST data. The RMSE of the maximum inundation depth at point-A is larger than the result in Fig. 6. This is because 4. Conclusions Fig. 6 includes the noninundated pixels in all the events, whereas In this study, we developed an emulator, Rain2Depth, of the point-A is an extremely inundated area. This indicates that the physical-based RRI model with a deep convolutional neural reconstructing in inundated areas has a difficultybyan emulator network. This network consists of feature extraction of the trained with the small data. The maximum inundation depth ob- rainfall data, feature transposition from the rainfall to the spa- tained using Rain2Depth is better than the result obtained using tial distribution of the maximum inundation depth, and recon- K20, possibly because of nonlinear transformation by the middle struction of the maximum inundation depth using spatial network in Rain2Depth (Table 3). For the total water volume in distribution features. To extract the rainfall and inundation fea- the target area, which has frequent inundation (Fig. 8), Rain2- tures, we used two approaches: PCA and CNN-AE. Because Depth with CNN-AE has the better agreement (Fig. 7b)witha CNN-AE can extract nonlinear features, the RMSE of the re- physical-based model than PCA because CNN-AE trained the constructed data with CNN-AE was smaller than the RMSE relationship with neighbor pixels by convolutional layers. with PCA. Thus, the dimensionality reduction by CNN-AE Figure 8 shows the example of the depth-MAP of TEST data was suitable for constructing an emulator of the physical-based obtained using Rain2Depth with PCA and CNN-AE. There is inundation around the Omono River, especially at serpentine RRI model while maintaining a small network trained with a and downstream locations (Fig. 8a). These inundation charac- small number of training data. teristics were retrieved by Rain2Depth with both PCA and Rain2Depth was constructed using a deep neural network to CNN-AE. The result shows Rain2Depth with CNN-AE had a connect the features of input and output variables. The results JANUARY 2023 MO M O I E T A L. 11 according to 5FCV indicated that Rain2Depth with CNN-AE is REFERENCES better than Rain2Depth with PCA when using the same num- Abe, S., S. Watanabe, M. Yamada, S. Kotsuki, and A. Watanuki, ber of features. In comparison with the previous study (K20), in 2019: Effects of precipitation observations and sea surface terms of maximum inundation depth at a specific location, we temperature patterns on inundation analysis using large-scale showed that Rain2Depth with CNN-AE was the best emulator climate prediction information (in Japanese with English among the three models because the middle layer in Rain2Depth abstract). J. Japan Soc. Civil Eng., 75,I_1081–I_1086, https:// provides nonlinear transformation. Thus, by using the middle doi.org/10.2208/jscejhe.75.2_I_1081. network in Rain2Depth (instead of K20) to emulate the maxi- Ba, J. L., J. R. Kiros, and G. E. Hinton, 2016: Layer normaliza- mum inundation depth at point A, we achieved better perfor- tion. arXiv, 1607.06450v1, https://arxiv.org/abs/1607.06450. Bocquet, M., A. Farchi, and Q. Malartic, 2021: Online learning of mance than K20 but worse performance than Rain2Depth both state and dynamics using ensemble Kalman filters. Found. with CNN-AE. Unlike K20, Rain2Depth can predict the spa- Data Sci., 3, 305–330, https://doi.org/10.3934/fods.2020015. tial distribution of maximum inundation depth. Chang, L.-C., H.-Y. Shen, Y.-F. Wang, J.-Y. Huang, and Y.-T. Lin, For future work, we have the three goals: evaluation of the 2010: Clustering-based hybrid inundation model for forecast- emulator through application to the actual inundation event ing flood inundation depths. J. Hydrol., 385,257–268, https:// (e.g., July 2017), optimization of the network with a small num- doi.org/10.1016/j.jhydrol.2010.02.028. ber of training data, and application to more complicated physics Gro ¨nquist, P., C. Yao, T. Ben-Nun, N. Dryden, P. Dueben, S. Li, processes (e.g., time series of the spatial distribution of inunda- and T. Hoefler, 2021: Deep learning for post-processing tion depth). Such improvements may enable the establishment of ensemble weather forecasts. Philos. Trans. Roy. Soc., A379, an operational flood warning system. 20200092, https://doi.org/10.1098/rsta.2020.0092. Japan suffers from an increasing trend of floods and inunda- Hashimoto, M., and T. Nakajima, 2017: Development of a remote tions induced by severe precipitation. Improving real-time sensing algorithm to retrieve atmospheric aerosol proper- inundation predictions is important to mitigate such disasters ties using multiwavelength and multipixel information. J. Geophys. Res. Atmos., 122,6347–6378, https://doi.org/10. from reservoir operations and provide sufficient time for resi- 1002/2016JD025698. dents’ evacuation. For that purpose, operational real-time sys- Hess, P., and N. Boers, 2022: Deep learning for improving numerical tems have been installed for major rivers in Japan using physical weather prediction of heavy rainfall. J. Adv. Model. Earth Syst., rainfall–runoff-inundation models with assimilation of water 14, e2021MS002765, https://doi.org/10.1029/2021MS002765. depth observation. There can be two possible applications to Hochreiter, S., and J. Schmidhuber, 1997: Long short-term memory. bring our methods to operational systems. Neural Comput., 9, 1735–1780, https://doi.org/10.1162/neco.1997. The first application is the use for computationally inexpensive 9.8.1735. inundation predictions. The second application is the use for the Ioffe, S., and C. Szegedy, 2015: Batch normalization: Accelerating data assimilation system of the RRI models. Because of the non- deep network training by reducing internal covariate shift. linearity of rainfall–runoff, river, and inundation processes, parti- arXiv, 1502.03167v3, https://arxiv.org/abs/1502.03167v3. cle filters would be a more suitable data assimilation method for Jhong, B.-C., J.-H. Wang, and G.-F. Lin, 2017: An integrated two- RRI models than ensemble Kalman filters and variational meth- stage support vector machine approach to forecast inundation maps during typhoons. J. Hydrol., 547, 236–252, https://doi.org/ ods. However, the particle filter requires many more ensembles 10.1016/j.jhydrol.2017.01.057. with more computations for employing RRI simulations. Here Keisler, R., 2022: Forecasting global weather with graph neural emulators could be used for increasing ensembles as augmented networks. arXiv, 2202.07575v1, https://doi.org/10.48550/arXiv. members (Kotsuki and Bishop 2022). In addition to such meth- 2202.07575. odological developments, tight discussions with operational Kikuchi, R., T. Misaka, and S. Obayashi, 2016: International journal administrations are also necessary. Such further updates toward of computational fluid dynamics real-time prediction of operational systems are important directions of our future studies. unsteady flow based on POD reduced-order model and particle filter. Int. J. Comput. Fluid Dyn., 30, 285–306, https://doi.org/10. Acknowledgments. This study was partly supported by the 1080/10618562.2016.1198782. Japan Society for the Promotion of Science (JSPS) KAKENHI Kiranyaz, S., R. Ince, R. Hamila, and M. Gabbouj, 2015: Convolu- Grants JP21J01854, JP21H04571, and 22H05230, JST PRESTO tional neural networks for patient-specificECG classification. 37th Annual Int. Conf. IEEE Engineering in Medicine and MJPR1924, Ministry of Education, Culture, Sports, Science and Biology Society (EMBC), Milan, Italy, IEEE, 2608–2611, Technology of Japan (JPMXP1020200305) under the “Program https://doi.org/10.1109/EMBC.2015.7318926. for Promoting Researches on the Supercomputer Fugaku” }}, O. Avci, O. Abdeljaber, T. Ince, M. Gabbouj, and D. J. (Large Ensemble Atmospheric and Environmental Prediction Inman, 2021: 1D convolutional neural networks and appli- for Disaster Prevention and Mitigation), the Ministry of Land, cations: A survey. Mech. Syst. Signal Process., 151, 107398, Infrastructure, Transportation and Tourism of Japan, and the https://doi.org/10.1016/j.ymssp.2020.107398. IAAR Research Support Program of Chiba University. Kobayashi, K., L. Duc, Apip, T. Oizumi, and K. Saito, 2020: Ensemble flood simulation for a small dam catchment in Data availability statement. Because of the large volume of Japan using nonhydrostatic model rainfalls}Part 2: Flood involved data and limited disk space available, the data will forecasting using 1600-member 4D-EnVar-predicted rain- be shared online upon request to author Masahiro Momoi falls. Nat. Hazards Earth Syst. Sci., 20,755–770, https://doi. (momoi-masahiro@doerresearch.com). org/10.5194/nhess-20-755-2020. 12 AR TI F I C I A L I N T E LLI G E N C E F O R T H E E AR TH S Y S T E M S VOLUME 2 Konja, A., Y. Nakamura, S. Abe, T. Sayama, and Y. Wakazuki, 2018: Pathak, J., and Coauthors, 2022: Fourcastnet: A global data-driven high-resolution weather model using adaptive Fourier neural Analysis of the flood control effect of the Kinugawa upper operators. arXiv, 2202.11214v1, https://doi.org/10.48550/arXiv. stream dam during the September 2015 Kanto-Tohoku heavy 2202.11214. rain (in Japanese with English abstract). J. Hydraul. Eng., 62, Ravuri, S., and Coauthors, 2021: Skilful precipitation nowcasting I_1507–I_1512, https://doi.org/10.2208/jscejhe.74.I_1507. using deep generative models of radar. Nature, 597, 672–677, Kotsuki, S., M. Momoi, R. Kikuchi, S. Watanabe, M. Yamada, https://doi.org/10.1038/s41586-021-03854-z. S. Abe, and A. Watanuki, 2020: Emulating rainfall-runoff- Sayama, T., G. Ozawa, T. Kawakami, S. Nabesaka, and K. Fukami, inundation model through ensemble learning of multiple 2012: Rainfall–runoff–inundation analysis of the 2010 Pakistan regularized regressors (in Japanese with English abstract). flood in the Kabul River basin. Hydrol. Sci. J., 57, 298–312, Ann. J. Hydraul. Eng., 76, 367–372, https://doi.org/10.2208/ https://doi.org/10.1080/02626667.2011.644245. jscejhe.76.2_I_367. Shi, C., M. Hashimoto, K. Shiomi, and T. Nakajima, 2020: Devel- }}, and C. H. Bishop, 2022: Implementing hybrid background opment of an algorithm to retrieve aerosol optical properties error covariance into the LETKF with attenuation-based over water using an artificial neural network radiative trans- localization: Experiments with a simplified AGCM. Mon. fer scheme: First result from GOSAT-2/CAI-2. IEEE Trans. Wea. Rev., 150, 283–302, https://doi.org/10.1175/MWR-D- Geosci. Remote Sens., 59, 9861–9872, https://doi.org/10.1109/ 21-0174.1. TGRS.2020.3038892. Krizhevsky, A., I. Sutskever, and G. E. Hinton, 2012: ImageNet Shi, X., Z. Gao, L. Lausen, H. Wang, D.-Y. Yeung, W.-K. Wong, classification with deep convolutional neural networks. Proc. and W.-C. Woo, 2017: Deep learning for precipitation nowcast- 25th Int. Conf. Neural Information Processing Systems,Vol. ing: A benchmark and a new model. Proc. 31st Int. Conf. on 1, Lake Tahoe, NV, Association for Computing Machinery, Neural Information Processing Systems, Vol. 30, Long Beach, 1097–1105, https://dl.acm.org/doi/10.5555/2999134.2999257. CA, Association for Computing Machinery, 5622–5632, https:// Liang, J., K. Terasaki, and T. Miyoshi, 2021: A machine learning dl.acm.org/doi/10.5555/3295222.3295313. approach to the observation operator for satellite radiance Takenaka, H., T. Y. Nakajima, A. Higurashi, A. Higuchi, T. data assimilation. 23rd EGU General Assembly,New Orleans, Takamura, R. T. Pinker, and T. Nakajima, 2011: Estimation LA, Amer. Geophys. Union, Abstract EGU21-10475, https:// of solar radiation using a neural network based on radiative doi.org/10.5194/egusphere-egu21-10475. transfer. J. Geophys. Res., 116, D08215, https://doi.org/10.1029/ Lin, G.-F., H.-Y. Lin, and Y.-C. Chou, 2013: Development of a 2009JD013337. real-time regional-inundation forecasting model for the inunda- Tsuyuki, T., and R. Tamura, 2022: Nonlinear data assimilation tion warning system. J. Hydroinf., 15, 1391–1407, https://doi.org/ by deep learning embedded in an ensemble Kalman filter. 10.2166/hydro.2013.202. J. Meteor. Soc. Japan, 100,533–553, https://doi.org/10. Lumley, J. L., 1967: The structure of inhomogeneous turbulent 2151/jmsj.2022-027. flows. Atmospheric Turbulence and Radio Wave Propagation, Van, S. P.,H.M. Le, D. V. Thanh, T. D. Dang,H.H.Loc,and A. M. Yaglom, Ed., Nauka, 166–178. D. T. Anh, 2020: Deep learning convolutional neural network Makinoshima, F., Y. Oishi, T. Yamazaki, T. Furumura, and in rainfall–runoff modelling. J. Hydroinf., 22, 541–561, https:// F. Imamura, 2021: Early forecasting of tsunami inundation doi.org/10.2166/hydro.2020.095. from tsunami and geodetic observation data with convolu- Watanabe, S., M. Yamada, S. Abe, and M. Hatono, 2020: Bias cor- tional neural networks. Nat. Commun., 12, 2253, https://doi. rection of d4PDF using a moving window method and their org/10.1038/s41467-021-22348-0. uncertainty analysis in estimation and projection of design rain- Mizuta, R., and Coauthors, 2017: Over 5,000 years of ensemble fall depth. Hydrol. Res. Lett., 14, 117–122, https://doi.org/10. future climate simulations by 60-km global and 20-km regional 3178/hrl.14.117. atmospheric models. Bull. Amer. Meteor. Soc., 98, 1383–1398, Wu, B., T. Zhou, and T. Li, 2009: Seasonally evolving dominant in- https://doi.org/10.1175/BAMS-D-16-0099.1. terannual variability modes of East Asian climate. J. Climate, Mosavi, A., P. Ozturk, and K.-W. Chau, 2018: Flood prediction 22, 2992–3005, https://doi.org/10.1175/2008JCLI2710.1. using machine learning models: Literature review. Water, 10, Yamazaki, D., S. Kanae, H. Kim, and T. Oki, 2011: A physically 1536, https://doi.org/10.3390/w10111536. based description of floodplain inundation dynamics in a Murray, N. E., and L. S. Ukeiley, 2007: An application of Gappy global river routing model. Water Resour. Res., 47, W04501, POD. Exp. Fluids, 42,79–91, https://doi.org/10.1007/s00348- https://doi.org/10.1029/2010WR009726. 006-0221-y. Yashiro, H., and Coauthors, 2020: A 1024-member ensemble data Nair, V., and G. E. Hinton, 2010: Rectified linear units improve assimilation with 3.5-km mesh global weather simulations. restricted Boltzmann machines. Proc. Int. Conf. Machine SC20: Int. Conf. for High Performance Computing, Network- Learning, Haifa, Israel, Association for Computing Machin- ing, Storage and Analysis, Atlanta, GA, IEEE, 1–10, https:// ery, 807–814, https://dl.acm.org/doi/10.5555/3104322.3104425. doi.org/10.1109/SC41405.2020.00005.

Journal

Artificial Intelligence for the Earth Systems – American Meteorological Society

Published: Jan 24, 2023

There are no references for this article.