Aiming at the index tracking problem in financial markets, a tracking model combining a novel non-convex RBS (robust, bounded and smooth) loss and group LASSO (least absolute shrinkage and selection operator) penalty is proposed. This model can select important constituent stocks in groups based on the convergence effect of stock volatility trends and construct a tracking model for the target index. It can achieve a lower tracking error even in the presence of abnormal data. Firstly, a novel non-convex RBS loss function is proposed to eliminate the influence of outliers. Secondly, the group LASSO penalty is introduced to group-penalize stocks with exhibiting volatile trends and convergence effects, thereby selecting important constituent stocks in groups based on the convergence effect. Finally, a regularization model is established with the objective of minimizing the tracking error, and the model is optimized and solved using the proximal gradient descent algorithm to obtain the important stocks and the corresponding tracking models. Through extensive numerical experiments, the robustness of the new method against outliers and its ability to select variables in groups are demonstrated. In the empirical analysis of tracking the SSE 50 Index, when the same number of important constituent stocks are selected, the new model achieves a lower tracking error compared to many well-known sparse portfolio methods, indicating its superior tracking ability and predictive performance.