目的 準確識別藏藥材，實現(xiàn)藏藥材智能化挖掘及管理。方法 提出基于超圖的雙模態(tài)特征融合藏藥材植株識別算法HerbiFusionNet模型。首先，利用改進的ResNet152-CA模型提取藏藥材圖像的空間特征，將基于Transformer架構(gòu)的BERT模型提取藏藥材文本的語義特征，實現(xiàn)2種模態(tài)特征的互補與融合；其次，計算融合后特征向量的相似性，構(gòu)建超圖網(wǎng)絡(luò)；最后，通過超圖神經(jīng)網(wǎng)絡(luò)捕獲藏藥材植株復雜關(guān)聯(lián)關(guān)系，獲得藏藥材準確的分類。結(jié)果 相比于單一模態(tài)ResNet-152-CA模型，引入融合雙模態(tài)特征并基于超圖神經(jīng)網(wǎng)絡(luò)的HerbiFusionNe模型，藏藥材識別準確率為96.28%，其準確率增加了4.40%。提出的HerbiFusionNet模型實證了融合圖像和文本的雙模態(tài)特征利用超圖結(jié)構(gòu)挖掘藏藥材數(shù)據(jù)內(nèi)復雜關(guān)系的有效性。結(jié)論 HerbiFusionNet模型提升了藏藥材識別的準確率，能有效捕捉藏藥材圖像與文本之間的高階關(guān)系，展現(xiàn)了超圖神經(jīng)網(wǎng)絡(luò)在處理藏藥植株復雜數(shù)據(jù)結(jié)構(gòu)中的優(yōu)勢，為后續(xù)深入挖掘“癥狀-方劑-藥材”關(guān)系及安全使用奠定了標準化基礎(chǔ)，推動了藏藥研究和應用的發(fā)展。

Objective To identify Tibetan medicinal materials accurately and realize the intelligent mining and management of Tibetan medicinal materials. Methods HerbiFusionNet, a hypergraph-based dual-modal feature fusion model for Tibetan medicinal plants recognition, was proposed. Firstly, the improved ResNet152-CA model was used to extract the spatial features of Tibetan medicinal materials images, and the Transformer architecture based BERT model was used to extract the semantic features of Tibetan medicinal materials texts to realize the complementarity and fusion of the two modal features. Then, the similarity of the fused feature vectors was calculated to construct a hypergraph network. Finally, the complex association relationship of Tibetan medicinal plants was captured by hypergraph neural network, and the accurate classification of Tibetan medicinal materials was obtained. Results The experimental results show that, compared with the single-modal ResNet-152-CA model, the HerbiFusionNe model based on hypergraph neural network introduced by fusing dual-modal features has an accuracy rate of 96.28%, which is increased by 4.40%. The HerbiFusionNet model proposed in this study demonstrates the effectiveness of using hypergraph structure to mine complex relationships in Tibetan medicinal materials data by fusing dual-modal features of image and text. Conclusion The HerbiFusionNet model improves the accuracy of Tibetan medicinal materials recognition, can effectively capture the high-order relationship between Tibetan medicinal materials images and texts, and shows the advantages of hypergraph neural network in dealing with the complex data structure of Tibetan medicinal plants. It lays a standardized foundation for further exploration of the “symptoms-prescription-medicinal materials” relationship and safe use, and promotes the development of Tibetan medicine research and application.

TP18;R282.5

青海省海南州可持續(xù)發(fā)展議程創(chuàng)新示范區(qū)科技創(chuàng)新平臺項目（2024-HN-P04）;中國高校產(chǎn)學研創(chuàng)新基金異構(gòu)智能計算專項（2024HY004）