An alternative approach is to design DL model architectures with improved efficiency. As one of the most widely-studied architectures in DL, numerous low-cost variants of CNNs have been proposed. Some popular examples of efficient CNN architectures include the MobileNet family136, DenseNet137, EfficientNet138, and CSPNet139. Similarly, numerous efficiency-based architectural modifications have been proposed for the transformer model, many of which aim to reduce the quadratic computational complexity incurred by the self-attention mechanism140. Additionally, some transformer architectural variants explore the use of parameter sharing and factorization to reduce the memory cost of model training141. Going further, efficient architectural variants have been discovered for RNNs142 and graph neural networks (GNNs)143,144, including specialized architectures that are tuned for better efficiency within the biological domain145.
!!INSTALL!! Rale Transformer Design Software 18
2ff7e9595c
Comments