ORIGINAL ARTICLE

Knowledge-enhanced graph neural networks for construction material quantity estimation of reinforced concrete buildings

Yifan Fei¹ | Wenjie Liao¹ | Xinzheng Lu¹ | Hong Guan²

1Key Laboratory of Civil Engineering Safety and Durability of China Education Ministry, Department of Civil Engineering, Tsinghua University, Beijing 100084, China

2School of Engineering and Built Environment, Griffith University, Gold Coast Campus, Queensland 4222, Australia

Correspondence
Xinzheng Lu, Department of Civil Engineering, Tsinghua University, Beijing, China
Email: luxz@tsinghua.edu.cn

Hong Guan, School of Engineering and Built Environment, Griffith University, Gold Coast Campus, Queensland, Australia
Email: h.guan@griffith.edu.au

Funding information

ABSTRACT

Computer-Aided Civil and Infrastructure Engineering, 2023

Download PDF Full Text

1        INTRODUCTION

Material consumption, as a critical component of the overall construction cost and carbon emissions, has been widely concerned in the design of building structures (Elfaki et al., 2014; Gan et al., 2020; Sharma et al., 2021). Contractors typically consider construction material quantity (CMQ) as an important aspect in selecting a structural design scheme that can achieve the minimum estimated overall construction cost. Furthermore, CMQ is directly related to the carbon emissions of the construction industry, which must be reduced to realize carbon neutrality and achieve the United Nations Sustainability Development Goals (SDGs) (UNEP, 2022). Although CMQ only affects a portion of the building budget and lifetime carbon footprint, it is the primary component that can be optimized through structural design. Hence, the design objectives of computer-aided structural design methods typically include CMQ minimization (Afzal et al., 2020; Málaga-Chuquitaype, 2022).

Reinforced concrete (RC) buildings are among the most widely used structural types being built with concrete and reinforcing steel. Steel consumption, which is relatively difficult to estimate, is a major concern for engineers in practical projects due to its significant impact on the construction cost. Concrete consumption is also gaining increasing attention nowadays after the SDGs become an urgent call for action by all countries. Currently, the mainstream CMQ estimation methods for RC buildings can be classified into three categories.

Category 1: Simplified formulas can be used to estimate CMQ (Sharafi et al., 2012; Zhang & Mueller, 2017; Zhou et al., 2022). In this category, the methods are highly efficient; however, they typically either display low accuracy or are only applicable to simple structural systems, such as frame structures.

Category 2: Structural design software, which is used to conduct detailed design, can also estimate the corresponding concrete and steel consumptions (Kim et al., 2013; El-Sokkary & Galal, 2020; Zhang et al., 2021). This method is the most accurate; however, it highly relies on a complicated physics-based analysis model and is therefore time-consuming and inefficient, normally requiring 1–10 minutes of calculation time for a typical RC building. Among computer-aided optimization design techniques, heuristic methods are usually recognized as effective and practical (Aldwaik & Adeli, 2014). The possibility of finding the optimal solution by heuristic methods is positively related to the number of evaluated solutions in the design space. Typically, heuristic methods require hundreds of evaluations during iterations before finding a rational solution (Adeli & Sarma, 2006; Boscardin et al., 2019; Zhang et al., 2023). Another kind of emerging design technique is artificial intelligence (AI)-based methods (Málaga-Chuquitaype, 2022). During the training of AI models, the prediction results must be evaluated to guide parameter updating. Due to the nature of big data, at least thousands of evaluations are needed for training. Consequently, with the current level of efficiency of structural design software in obtaining the CMQs of RC buildings, design and optimization requirements are difficult to meet. To reduce the computational cost, surrogate models have been proposed, as discussed in the next category.

Category 3: The CMQ can be estimated using AI-based or data-driven methods (Idowu & Lam, 2020; Antoniou et al., 2023; Kovačević et al., 2023), which allows a good balance to be achieved between accuracy and efficiency. They have been widely used as surrogate models to evaluate the objective functions of evolutionary algorithms (Jin et al., 2019; Chugh et al., 2019). The foregoing is the focus of this study as elaborated in the following sections.

The fundamental paradigm of data-driven CMQ estimation methods lies in three aspects (Singh, 1991; Yeh, 1998; Son et al., 2013; Fragkakis et al., 2015; García de Soto et al., 2014; Dimitriou et al., 2018; Beljkaš et al., 2020; Idowu & Lam, 2020; Kovačević et al., 2023; Antoniou et al., 2023). (1) Real-world engineering cases are collected, and CMQs are used as label data. (2) Key features are manually selected as input data. (3) A data-driven model is selected to establish the mapping relationship between inputs and labels. However, the existing research on this paradigm has the following limitations: (a) The number of engineering cases collected is typically limited; therefore, the generalization of the model requires further exploration. (b) CMQ is affected by heterogeneous data such as component layout (topology), component section size (geometry), and design conditions (text). However, due to the limitations of their model architecture, existing methods cannot directly handle this data and require manual feature selection, which is inevitably restricted by the knowledge and experience of the developers. (c) A data-driven model is a “black box”, which is not guided by prior knowledge; consequently, it is prone to fundamental errors.

In recent years, intelligent structural design methods based on generative AI have developed rapidly (Liao et al., 2021; Fu et al., 2023). This technology can quickly generate numerous structural designs and provide valuable data that can be applied to real-world engineering cases. Deep learning models can automatically mine high-dimensional hidden features from data, thereby avoiding the subjectivity of manually defined input features. In addition, in previous research, the authors have successfully integrated engineering knowledge into deep learning, effectively improving the reliability of data-driven models (Fei et al., 2022a; Lu et al., 2022). These previous studies have established a solid foundation for the present study.

Specifically, this study proposes to establish the following strategies to solve the problems that have been encountered by the existing studies. (1) A data augmentation method is proposed, incorporating generative adversarial networks (GANs) and parametric modeling. A considerable amount of synthetic data is generated, and therefore dataset diversity is enriched. (2) A graph neural network (GNN) architecture is proposed herein, which enables direct mapping from all related information to CMQs through a heterogeneous feature fusion mechanism, thereby curtailing the limitations of manually selected features. (3) A prior knowledge inclusion strategy is proposed, which avoids fundamental errors by modifying the output layer and the loss function of the GNNs.

To validate the effectiveness of the proposed methods, RC shear wall building structures, typically used in high-rise residential buildings, are considered as practical examples. The CMQs of these buildings are difficult to estimate because of the presence of shear walls, but are a key factor that must be considered in structural design. Accordingly, the use of a data-driven method to accurately and efficiently estimate the CMQs of RC shear wall building structures is desirable.

The remainder of this paper is organized as follows. Section 2 reviews related studies. Section 3 presents the outline of a novel CMQ estimation method for RC buildings. Section 4 introduces the knowledge-enhanced GNN specifically designed for CMQ estimations. Section 5 describes the data augmentation of the CMQ dataset based on GANs and parametric modeling. Section 6 introduces three other data-driven methods for comparison. Section 7 presents the numerical experiments. Section 8 discusses the case study of several shear wall structures of residential buildings. Finally, the conclusions of this study are presented in Section 9.

2        REVIEW OF CURRENT LITERATURE

2.1  Data-driven CMQ estimation

The research on CMQ estimation using data-driven methods can be traced back to the 1990s (Singh, 1991), and continues to advance recently (Antoniou et al., 2023). The common methods used in relevant research include artificial neural network (ANN) (Adeli & Wu, 1998; Dimitriou et al., 2018; Beljkaš et al., 2020), support vector machine (Idowu & Lam, 2020), and regression analysis (Fragkakis et al., 2015; Son et al., 2013). The research targets include buildings (García de Soto et al., 2014; Idowu & Lam, 2020), bridges (Beljkaš et al., 2020; Kovačević et al., 2023), and civil infrastructures (Fragkakis et al., 2015; Antoniou et al., 2023).

Similarly, a significant amount of research is dedicated to estimating construction costs (Adeli & Wu, 1998; Rafiei & Adeli, 2018). These estimation methods are widely used in the cost optimization of concrete structures (Ahmadkhanlou & Adeli, 2005; Aldwaik & Adeli, 2016), steel structures (Sarma & Adeli, 2000a; Sarma & Adeli, 2000b), and composite structures (Kim & Adeli, 2001; Adeli & Kim, 2001). This study mainly focuses on the CMQ estimation because the material cost can be easily calculated based on CMQ if the material price is provided.

Table 1 summarizes some related studies on CMQ estimation of RC structures, including the target, method, number of variables, data size, and prediction error measured by the mean absolute percentage error (MAPE) (if applicable). These studies support the decision-making of stakeholders of construction projects by estimating CMQs. These estimations can be used in design optimization as long as the design variables of RC structures are included as the input variables of the data-driven model. The ground truth of the CMQs is obtained from the design documents, especially the bill of quantities. The accuracy of existing methods (2% ≤ MAPE ≤ 30%) is influenced by many factors and is in need of further improvement. These factors include: (1) only a small number of manually selected variables (three to ten) are considered and the methods may have overlooked useful information that developers are not aware of; (2) data-driven models are prone to fundamental errors if no prior knowledge is incorporated; (3) the data size (or the number of different RC structure samples) is small (6 to 400) compared to those of nowadays’ big data tasks (Sun et al., 2017), and the sample diversity is limited. The above-mentioned factors are further discussed in Sections 2.2, 2.3, and 2.4, respectively.

TABLE 1. Related studies on CMQ estimation of RC structures

Note: RA=Regression Analysis; ANN=Artificial Neural Networks; GPR=Gaussian Process Regression; SVM=Support Vector Machine; MAPE=Mean Absolute Percentage Error; N/A=Not Applicable.

2.2   Graph neural networks

With the development of deep learning technology, deep neural networks can automatically extract key features from massive amounts of information. Deep learning approaches are gradually replacing traditional feature engineering methods in the fields of computer vision and natural language processing (Russakovsky et al., 2015; Bubeck et al., 2023).

GNN is a type of deep learning approach that operates on graphs, a data structure that represents a set of objects (nodes) and their relationships (edges) (Zhou et al., 2020). In recent years, variants of GNNs such as graph convolutional networks (GCN) (Kipf & Welling, 2016) and Graph-SAGE (Hamilton et al., 2018) have exhibited excellent performances on many deep learning tasks. Nowadays, the applications of GNN have been extended to various tasks with graph-like data structures, such as building structures (Song et al., 2022; Zhao et al., 2023a), water distribution networks (Fan et al., 2023), package-based constraint management (Wu et al., 2023), autonomous vehicle networks (Chen et al., 2021), neural network architectures (Xue et al., 2021), neural signals (Feng et al., 2021), and electroencephalogram signals (Zhao et al., 2021; Che et al., 2022; Lian & Xu, 2022).

However, existing GNNs cannot be directly applied to CMQ estimation because of the heterogeneous input data of this task, including component layout (topology), component section sizes (geometry), and design conditions (text). Hence, this study proposes a novel GNN architecture with a heterogeneous feature fusion mechanism, so that all related information can be directly input into the model without specifying any features artificially.

2.3  Knowledge-enhanced neural networks

Two principal rules must be satisfied to achieve the CMQ estimation of RC buildings based on prior knowledge in structural engineering practice. (1) The total concrete consumption of a building structure is less than the sum of the calculated volumes of all structural components. This is due to the common duplications in calculating the volumes of components. For example, doubling the concrete volume of overlapped beam–column joints must be avoided. (2) The amount of steel reinforcement for any structural component must satisfy the minimum requirements specified in the design codes (MOHURD, 2010, 2015, 2016). However, to the best knowledge of the authors, such prior knowledge has not been considered in the existing data-driven CMQ estimation methods. Hence, unexpected major and fundamental errors may occur during the data-driven model estimation process considering the statistical nature of such models.

Generally, for neural networks, prior knowledge is introduced by modifying the following three elements of the network (Dash et al., 2022): (1) input data (Yin et al., 2019; Marino et al., 2021), (2) loss function (Lu et al., 2022; Fei et al., 2022a; Du et al., 2022), and (3) network architecture (Sourek et al., 2018; Du et al., 2020; Li et al., 2021). The second element, viz., modifying the loss function, is adopted in this study. The output layer of the neural networks is also modified to ensure 100% satisfaction of the prior knowledge.

2.4  Data augmentation of structural design

Table 1 indicates that the data sizes used in the existing studies are no larger than 400, which is much smaller than those of typical deep learning tasks (Sun et al., 2017). This may result in a limited generalization of trained models and difficulty in applying them to practical projects. Since collecting large-scale real-world engineering project data is costly or even unattainable under certain circumstances (D’Amico et al., 2019), the use of synthetic data generated by generative AI to supplement real-world data is desirable.

Structural design methods based on AI have emerged rapidly in recent years and have received extensive attention (Adeli, 2020; Sun et al., 2021; Málaga-Chuquitaype, 2022). The AI technologies used in relevant research projects include GANs (Zhang et al., 2021; Khan et al., 2023), GNNs (Wang & Zhang, 2021; Chang & Cheng, 2020), convolutional neural networks (CNNs) (Yu et al., 2019; Ampanavos et al., 2022), and reinforcement learning (Hayashi & Ohsaki, 2021; Jeong & Jo, 2021). In particular, a series of research studies have focused on GAN-based design methods for building structures, providing a good foundation for this study (Pizarro et al., 2021; Liao et al., 2021; Fu et al., 2023).

The GAN approach is also a well-recognized data augmentation method (Shorten & Khoshgoftaar, 2019) that can generate realistic synthetic data. However, it has yet to be applied to CMQ estimation tasks because of the difficulty in automatically obtaining the CMQ of synthetic structural design. Therefore, this study proposes a data augmentation method, combining parametric modeling with the GAN-based design method, thereby increasing the CMQ data size by an order of magnitude.

3        Outline of the proposed CMQ estimation method

This study aims to resolve the problems encountered in the CMQ estimation of RC buildings. Consider an RC shear wall building structure as an example. Given the actual structural design (including the spatial layout and geometric dimensions of shear walls and beams) and design conditions (including seismic, site, and height conditions), the concrete and steel consumptions (of shear walls, beams, and slabs) are estimated. The CMQ estimation method proposed in this study, including three modules, is shown in Figure 1.

(1) Model application module: The structural design and design conditions to be considered are represented as a graph (Section 4.1). The concrete and steel consumptions are estimated by inputting the graph into a trained GNN model (Section 4.2). The GNN parameters are generated from Module (2).

(2) Model training module: The GNN model is trained based on the CMQ dataset. The model can extract and fuse Global, Edge, and Node features, and is thereby named as Graph-GEN. It can also embed prior knowledge to ensure that the CMQ estimation satisfies the principal rules of not exceeding the maximum concrete consumption and not below the minimum steel consumption (Section 4.3). The CMQ dataset, including input and label data, is generated from Module (3).

(3) Data augmentation module: A GAN model is trained using a dataset collected from real-world engineering projects. It is then used to generate a large number of synthetic structural designs corresponding to synthetic design conditions. After that, refined finite element models are established using parametric modeling, and their CMQs are determined. This process expands the size of the CMQ dataset by an order of magnitude, thus facilitating the effective training of Graph-GEN (Section 5).

4        Knowledge-enhanced GNNs

A novel GNN architecture called Graph-GEN is proposed that incorporates a heterogeneous feature fusion mechanism and a prior knowledge inclusion strategy. It is capable of extracting and fusing high-dimensional features in various forms, including component layout (topology), component section sizes (geometry), and design conditions (text), which are subsequently mapped into CMQs. Innovatively, prior knowledge is introduced into Graph-GEN by modifying the loss function and output layer.

Figure 1. The proposed CMQ estimation method

4.1    Feature representation – model application module

To use a GNN for CMQ estimation, the structural design of an RC building must be represented as a graph. A straightforward approach is adopted to represent the topological relationship of structural components. It is applicable to walls, columns, beams, braces, and so forth, where the structural components are represented by graph edges and the intersections of the structural components are represented by graph nodes (Zhao et al., 2023b). Among all the floors of a building structure, a standard floor can best represent the planar layout of structural components. Therefore, a standard floor of a shear wall building structure is considered herein as an example. The planar layout of the shear walls and beams is represented as a graph, as shown in Figure 2, in which the coordinates of the intersections are used as node features, and the geometric dimensions of the structural components are used as edge features. Thus, all the structural design information can be represented as a graph. For this standard floor of a shear wall building structure, the node features are two-dimensional vectors containing the X and Y coordinates, and the edge features are three-dimensional vectors comprising the length, width, and height of the structural components, as described in Table 2.

Design conditions also significantly affect the CMQ estimation of a building structure (MOHURD, 2010). For example, if a building structure is subjected to a large horizontal seismic action, the amount of reinforcement of the vertical components on the ground floor must increase accordingly. For shear wall building structures, three design conditions are selected for illustration, including seismic, site, and height design conditions, which are represented by the design seismic acceleration ( ), the site characteristic period ( ), and the number of floors ( ), respectively (MOHURD, 2016). Further details are provided in Appendix A. The foregoing design conditions are also represented as a three-dimensional vector known as a global feature because they affect the CMQs of all structural components, as shown in Table 2. The global feature is highly scalable, and its size can be adjusted to consider various design conditions.

The CMQs to be predicted are represented as six-dimensional or two-dimensional vectors comprising the concrete and steel consumptions of shear walls, beams, and slabs (separated or combined), as given in Table 2. These two representations will be discussed in Section 7.2.

TABLE 2. Input and output variables of Graph-GEN

FIGURE 2. Graph representation of shear wall building structure: (a) planar layout of shear walls and beams; (b) graph nodes and edges

Figure 3. Graph-GEN architecture

4.2    Model architecture - Graph-GEN

Given the fact that both the structural design (represented by a graph) and design conditions (represented by a vector) significantly affect the CMQs, this study therefore proposes a novel GNN architecture, named Graph-GEN, which extracts and fuses Global, Edge, and Node features, as demostrated in Figure 3.

Firstly, each node representation is generated by aggregating the representations of its adjacent nodes and the features of its incident edges, the intuitive meaning of which is the CMQ requirements of structural joints (Algorithm 1 in Figure 3). Secondly, each edge prediction is generated using the representations of itself and its end nodes, with an intuitive meaning being the CMQ in each structural component. All edge predictions are then summed up into a graph representation, intuitively standing for an initial CMQ estimation (Algorithm 2 in Figure 3). Thirdly, the processed global features (design conditions) and the graph representation are concatenated and mapped into the final CMQ estimation (Algorithm 3 in Figure 3). The core algorithms with matching steps are introduced below, and the input and output features have been described in Section 4.1.

Step 1: Input the graph , node features , and edge features  into the GNN layers to generate the node representations . The GNN layer is constructed upon Graph-SAGE (Hamilton et al., 2018) and improved to consider the edge features (geometric dimensions of the structural components) (Zhao et al., 2023a).

Algorithm 1 describes the feature aggregation process of Graph-GEN (Figure 3). For the k-th GNN layer, each node first concatenates the current node representations , the edge features , and the neighbor node representations . Then, the concatenated vector is fed into a fully connected layer with non-linear activation and mapped into a temporary vector. After that, all temporary vectors in the immediate neighborhood are summed up into a single vector . This vector is concatenated with the node’s current representation . Note that for the first GNN layer, the node representation  is the input node feature. The concatenated vector is again fed into a fully connected layer with non-linear activation and mapped into an updated node representation  to be used in the next GNN layer.

Algorithm 1. Feature aggregation of Graph-GEN

Step 2: Based on the updated node representations  and non-updated edge features , the edge predictions and the graph representation  can be obtained (Figure 3). Specifically, this is accomplished by concatenating the edge feature and end node representations of each edge, and passing the concatenated vector to fully connected layers with non-linear activation to obtain the edge prediction  (Zhao et al., 2023a). Since this operation is performed on the edge level, the size of the weight matrices is irrelevant to the size of the input graph. Subsequently, all edge predictions are summed into one vector and passed onto the fully connected layers with non-linear activation to obtain the graph representation y. Summing all edge predictions ensures that the size of the graph representation y is consistent and irrelevant to the size of the input graph. This approach is intuitive. First, the CMQs of each structural component are estimated by simultaneously considering the structural component itself (edge feature) and the joints at both of its ends (end node representations). Then, the preliminary CMQs of the entire building structure are obtained by summing those of all structural components. Algorithm 2 below provides further details. Note that Algorithm 2 is different from the existing methods in its unique way to obtain the graph representation, considering all edge and node features.

Algorithm 2. Global representation of Graph-GEN

Step 3: The CMQs can be estimated based on the graph representation  and the global feature . Specifically, the global feature is inputted into the fully connected layers with non-linear activation, and the output vector is concatenated with the graph representation. The concatenated vector is then inputted into the fully connected layers with non-linear activation and mapped as the CMQ representation  (Figure 3). The intuitive attribute of this approach is to use the global features (design conditions) to adjust the graph representations (preliminary CMQs). Further details are provided in Algorithm 3 below. Note that the novelty of Algorithm 3 lies in its separate processing of the global feature and effective fusion of heterogeneous features.

Details of Graph-GEN architecture, including the specific setting of each neural network layer, can be found in Appendix B. Three architecture settings are presented with different graph depth K, fully connected layer depths M and N, and are later discussed in Section 7.1.

Algorithm 3. Feature fusion of Graph-GEN

4.3    Knowledge inclusion - model training module

In the CMQ estimation task for RC buildings, the following prior knowledge described in Section 2.3 is elaborated herein.

(1) The total concrete consumption of the building structure is less than the sum of the calculated volumes of all structural components. This is apparent because an inevitable duplication exists in calculating the volumes of structural components. The concrete volume of the overlapped regions (e.g., beam–column and beam–wall joints) must not be doubly counted.

(2) The steel consumption of any structural component must satisfy the minimum reinforcement requirements as specified in the design codes (MOHURD, 2010, 2015, 2016). Evidently, this prior knowledge must be determined according to the local codes. For the RC shear wall building structures constructed in China, the minimum reinforcement requirements for shear walls, beams, and slabs are presented in Appendix C.

This study introduces the foregoing prior knowledge into data-driven models by modifying the loss function and output layer. Specifically, to ensure that the prior knowledge is 100% satisfied, knowledge transformations are performed at the output layer, and a knowledge inclusion loss function ( ) is derived based on these transformations (Dash et al., 2022). This approach is applicable not only to the proposed Graph-GEN but also to other neural network-based models. The loss function of such knowledge-enhanced neural networks is expressed by Equations (1)–(3):

where  is the loss function of knowledge-enhanced neural networks;  is the data-driven loss function;  is the knowledge inclusion loss function;  is the mean square error loss function;  returns the zero tensor of the same shape as the input tensor; and  is the weight of knowledge loss ( );  and  are the labels of concrete and steel consumptions, respectively.

 and  are the concrete and steel consumptions after the transformations, as given by Equations (4) and (5), respectively.

where  is the weight of knowledge transformation ( );  and  are the concrete and steel consumptions directly outputted by the neural networks, respectively;  and  are the portions of concrete and steel consumptions that do not conform to the prior knowledge, as given by Equations (6) and (7), respectively.

where  is the rectified linear unit activation function;  and  are the adjusting factors obtained from datasets, as given by Equations (8) and (9).

where  and  are the labels for concrete and steel consumptions of the i-th sample, respectively;  and  are the maximum concrete and minimum steel consumptions of the i-th sample, respectively; N is the number of samples in the dataset; Min{·} and Max{·} return the minimum and maximum elements in the list, respectively.

 and  are the maximum concrete and minimum steel consumptions calculated according to prior knowledge, as given by Equations (10) and (11) (for RC shear wall building structures), respectively.

where , , and  are the calculated volumes of a single shear wall, beam, and slab, respectively (the steel volume is neglected); , , and  are the minimum steel consumptions of a single shear wall, beam, and slab, respectively, calculated according to the minimum reinforcement requirements given in Appendix C.

Note that the values of  and  have significant impacts on the model performance, especially in extreme cases when  or ; this is discussed in Section 7.3.

5        Data augmentation module

The performance and generalization of data-driven models are closely related to their data size. As mentioned in Section 2.1, collecting a number of real-world engineering design cases much larger than 400 for CMQ estimation is rather challenging. Therefore, this study proposes a data augmentation method based on GAN and parametric modeling to expand the size of the CMQ dataset by an order of magnitude. First, the GAN is trained based on real-world design data. Subsequently, a large amount of synthetic design data is generated using GAN. Finally, the CMQs corresponding to the synthetic design data are obtained using parametric modeling. To provide a specific explanation of the foregoing, consider an RC shear wall building structure again as an example.

5.1    Structural design dataset

To train the GAN, a structural design dataset containing 430 input data-output data pairs is collected from real-world engineering projects involving shear wall structures of residential buildings, as illustrated in Figure 4. Note that the data size (430) will be increased following data augmentation.

Each data pair includes input and label data. The input data refer to the architectural design (planar layout of partition walls, doors, and windows) and design conditions ( , , and , as described in Section 4.1). The label data denote the structural design (planar layout of shear walls, which is critical in the schematic design of shear wall building structures). The architectural and structural designs are represented as pixel images, and the design conditions are represented as text (Liao et al., 2022). The real-world dataset is split into two parts: 80% of the dataset is used for training and validation, and 20% is set aside for testing.