A journal of IEEE and CAA , publishes high-quality papers in English on original theoretical/experimental research and development in all areas of automation
Volume 9 Issue 7
Jul.  2022

IEEE/CAA Journal of Automatica Sinica

  • JCR Impact Factor: 11.8, Top 4% (SCI Q1)
    CiteScore: 23.5, Top 2% (Q1)
    Google Scholar h5-index: 77, TOP 5
Turn off MathJax
Article Contents
Y.-B. Wang, J.-Y. Hang, and M.-L. Zhang, “Stable label-specific features generation for multi-label learning via mixture-based clustering ensemble,” IEEE/CAA J. Autom. Sinica, vol. 9, no. 7, pp. 1248–1261, Jul. 2022. doi: 10.1109/JAS.2022.105518
Citation: Y.-B. Wang, J.-Y. Hang, and M.-L. Zhang, “Stable label-specific features generation for multi-label learning via mixture-based clustering ensemble,” IEEE/CAA J. Autom. Sinica, vol. 9, no. 7, pp. 1248–1261, Jul. 2022. doi: 10.1109/JAS.2022.105518

Stable Label-Specific Features Generation for Multi-Label Learning via Mixture-Based Clustering Ensemble

doi: 10.1109/JAS.2022.105518
Funds:  This work was supported by the National Science Foundation of China (62176055) and the China University S&T Innovation Plan Guided by the Ministry of Education
More Information
  • Multi-label learning deals with objects associated with multiple class labels, and aims to induce a predictive model which can assign a set of relevant class labels for an unseen instance. Since each class might possess its own characteristics, the strategy of extracting label-specific features has been widely employed to improve the discrimination process in multi-label learning, where the predictive model is induced based on tailored features specific to each class label instead of the identical instance representations. As a representative approach, LIFT generates label-specific features by conducting clustering analysis. However, its performance may be degraded due to the inherent instability of the single clustering algorithm. To improve this, a novel multi-label learning approach named SENCE (stable label-Specific features gENeration for multi-label learning via mixture-based Clustering Ensemble) is proposed, which stabilizes the generation process of label-specific features via clustering ensemble techniques. Specifically, more stable clustering results are obtained by firstly augmenting the original instance repre-sentation with cluster assignments from base clusters and then fitting a mixture model via the expectation-maximization (EM) algorithm. Extensive experiments on eighteen benchmark data sets show that SENCE performs better than LIFT and other well-established multi-label learning algorithms.


  • loading
  • [1]
    M.-L. Zhang and Z.-H. Zhou, “A review on multi-label learning algorithms,” IEEE Trans. Knowledge and Data Engineering, vol. 26, no. 8, pp. 1819–1837, 2014. doi: 10.1109/TKDE.2013.39
    A. McCallum, “Multi-label text classification with a mixture model trained by EM,” in Proc. Working Notes AAAI’99 Workshop Text Learning, Orlando, FL, 1999.
    M. R. Boutell, J. Luo, X. Shen, and C. M. Brown, “Learning multi-label scene classification,” Pattern Recognition, vol. 37, no. 9, pp. 1757–1771, 2004. doi: 10.1016/j.patcog.2004.03.009
    H. Kazawa, T. Izumitani, H. Taira, and E. Maeda, “Maximal margin labeling for multi-topic text categorization,” in Proc. 17th Int. Conf. Neural Information Processing Systems, Vancouver, Canada, 2004, pp. 649–656.
    Z. Barutcuoglu, R. E. Schapire, and O. G. Troyanskaya, “Hierarchical multi-label prediction of gene function,” Bioinformatics, vol. 22, no. 7, pp. 830–836, 2006. doi: 10.1093/bioinformatics/btk048
    M.-L. Zhang and L. Wu, “LIFT: Multi-label learning with labelspecific features,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 37, no. 1, pp. 107–120, 2014.
    A. Topchy, J. A. K, and W. Punch, “A mixture model for clustering ensembles,” in Proc. SIAM Int. Conf. Data Mining, Florida, USA, 2004, pp. 379–390.
    W. Zhan and M.-L. Zhang, “Multi-label learning with label-specific features via clustering ensemble,” in Proc. IEEE Int. Conf. Data Science and Advanced Analytics, Tokyo, Japan, 2017, pp. 129–136.
    Z.-H. Zhou and W. Tang, “Clusterer ensemble,” Knowledge-Based Systems, vol. 19, no. 1, pp. 77–83, 2006. doi: 10.1016/j.knosys.2005.11.003
    H. Ayad and M. Kamel, “Finding natural clusters using multi-clusterer combiner based on shared nearest neighbors,” in Proc. 4th Int. Workshop Multiple Classifier Systems, Surrey, UK, 2003, pp. 166–175.
    E. Gibaja and S. Ventura, “A tutorial on multilabel learning,” ACM Computing Surveys, vol. 47, no. 3, pp. 1–38, 2015.
    M.-L. Zhang, Y.-K. Li, Y.-Y. Liu, and X. Geng, “Binary relevance for multi-label learning: An overview,” Frontiers of Computer Science, vol. 12, no. 2, pp. 191–202, 2018. doi: 10.1007/s11704-017-7031-7
    C. Brinker, E. Loza Mencía, and J. Fürnkranz, “Graded multilabel classification by pairwise comparisons,” in Proc. 14th IEEE Int. Conf. Data Mining, Shenzhen, China, 2014, pp. 731–736.
    J. Fürnkranz, E. Hüllermeier, E. Loza Mencía, and K. Brinker, “Multilabel classification via calibrated label ranking,” Machine Learning, vol. 73, no. 2, pp. 133–153, 2008. doi: 10.1007/s10994-008-5064-8
    J. Read, B. Pfahringer, G. Holmes, and E. Frank, “Classifier chains for multi-label classification,” Machine Learning, vol. 85, no. 3, pp. 333–359, 2011. doi: 10.1007/s10994-011-5256-5
    G. Tsoumakas, I. Katakis, and I. Vlahavas, “Random k-labelsets for multi-label classification,” IEEE Trans. Knowledge and Data Engineering, vol. 23, no. 7, pp. 1079–1089, 2011. doi: 10.1109/TKDE.2010.164
    M. Huang, F. Zhuang, X. Zhang, X. Ao, Z. Niu, M.-L. Zhang, and Q. He, “Supervised representation learning for multi-label classification,” Machine Learning, vol. 108, no. 5, pp. 747–763, 2019. doi: 10.1007/s10994-019-05783-5
    L. Sun, S. Ji, and J. Ye, Multi-Label Dimensionality Reduction. Boca Ration, FL: Chapman and Hall/CRC, 2013.
    C. Yan, X. Chang, M. Luo, Q. Zheng, X. Zhang, Z. Li, and F. Nie, “Selfweighted robust lda for multiclass classification with edge classes,” ACM Trans. Intelligent Systems and Technology, vol. 12, no. 1, pp. 1–19, 2020.
    X. Chang, F. Nie, S. Wang, Y. Yang, X. Zhou, and C. Zhang, “Compound rank-k projections for bilinear analysis,” IEEE Trans. Neural Networks and Learning Systems, vol. 27, no. 7, pp. 1502–1513, 2015.
    R. B. Pereira, A. Plastino, B. Zadrozny, and L. H. C. Merschmann, “Categorizing feature selection methods for multi-label classification,” Artificial Intelligence Review, vol. 49, no. 1, pp. 57–78, 2018. doi: 10.1007/s10462-016-9516-4
    Z. Cai and W. Zhu, “Feature selection for multi-label classification using neighborhood preservation,” IEEE/CAA J. Autom. Sinica, vol. 5, no. 1, pp. 320–330, 2017.
    C. Yan, Q. Zheng, X. Chang, M. Luo, C.-H. Yeh, and A. G. Hauptman, “Semantics-preserving graph propagation for zero-shot object detection,” IEEE Trans. Image Processing, vol. 29, pp. 8163–8176, 2020. doi: 10.1109/TIP.2020.3011807
    Y. Chen, X. Yang, J. Li, P. Wang, and Y. Qian, “Fusing attribute reduction accelerators,” Information Sciences, vol. 587, pp. 354–370, 2022. doi: 10.1016/j.ins.2021.12.047
    S. Canuto, M. A. Gonçalves, and F. Benevenuto, “Exploiting new sentiment-based meta-level features for effective sentiment analysis,” in Proc. 9th ACM Int. Conf. Web Search and Data Mining, San Francisco, CA, 2016, pp. 53–62.
    Y. Yang and S. Gopal, “Multilabel classification with meta-level features in a learning-to-rank framework,” Machine Learning, vol. 88, no. 1−2, pp. 47–68, 2012. doi: 10.1007/s10994-011-5270-7
    X. Wu, Q.-G. Chen, Y. Hu, D. Wang, X. Chang, X. Wang, and M.-L. Zhang, “Multi-view multi-label learning with view-specific information extraction,” in Proc. 28th Int. Joint Conf. Artificial Intelligence, Macau, China, 2019, pp. 3884–3890.
    C. Zhang, Z. Yu, Q. Hu, P. Zhu, X. Liu, and X. Wang, “Latent semantic aware multi-view multi-label classification,” in Proc. 32nd AAAI Conf. Artificial Intelligence, New Orleans, LA, 2018, pp. 4414–4421.
    W. Zhan and M.-L. Zhang, “Inductive semi-supervised multi-label learning with co-training,” in Proc. 23rd ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, 2017, pp. 1305–1314.
    J. Huang, G. Li, Q. Huang, and X. Wu, “Learning label-specific features and class-dependent labels for multi-label classification,” IEEE Trans. Knowledge and Data Engineering, vol. 28, no. 12, pp. 3309–3323, 2016. doi: 10.1109/TKDE.2016.2608339
    W. Weng, Y. Chen, C. Chen, S. Wu, and J. Liu, “Non-sparse label specific features selection for multi-label classification,” Neuro-computing, vol. 377, pp. 85–94, 2020. doi: 10.1016/j.neucom.2019.10.016
    X.-Y. Jia, S.-S. Zhu, and W.-W. Li, “Joint label-specific features and correlation information for multi-label learning,” J. Computer Science and Technology, vol. 35, no. 2, pp. 247–258, 2020. doi: 10.1007/s11390-020-9900-z
    J. Zhang, C. Li, D. Cao, Y. Lin, S. Su, L. Dai, and S. Li, “Multi-label learning with label-specific features by resolving label correlations,” Knowledge-Based Systems, vol. 159, pp. 148–157, 2018. doi: 10.1016/j.knosys.2018.07.003
    L. Sun, M. Kudo, and K. Kimura, “Multi-label classification with meta-label-specific features,” in Proc. 23rd Int. Conf. Pattern Recognition, Cancun, Mexico, 2016, pp. 1612–1617.
    S. Xu, X. Yang, H. Yu, D.-J. Yu, J. Yang, and E. C. C. Tsang, “Multilabel learning with label-specific feature reduction,” Knowledge-Based Systems, vol. 104, pp. 52–61, 2016. doi: 10.1016/j.knosys.2016.04.012
    W. Weng, Y. Lin, S. Wu, Y. Li, and Y. Kang, “Multi-label learning based on label-specific features and local pairwise label correlation,” Neurocomputing, vol. 273, pp. 385–394, 2018. doi: 10.1016/j.neucom.2017.07.044
    J. Ma, H. Zhang, and T. W. S. Chow, “Multilabel classification with label-specific features and classifiers: A coarse-tuned and fine-tuned framework,” IEEE Trans. Cybernetics, vol. 51, no. 2, pp. 1028–1042, 2019.
    Z.-B. Yu and M.-L. Zhang, “Multi-label classification with label-specific feature generation: A wrapped approach,” IEEE Trans. Pattern Analysis and Machine Intelligence, 2021.
    M.-L. Zhang, J.-P. Fang, and Y.-B. Wang, “Bilabel-specific features for multi-label classification,” ACM Trans. Knowledge Discovery from Data, vol. 16, no. 1, pp. 1–23, 2021.
    Z.-S. Chen and M.-L. Zhang, “Multi-label learning with regularization enriched label-specific features,” in Proc. 11th Asian Conf. Machine Learning, Nagoya, Japan, 2019, pp. 411–424.
    C.-Y. Zhang and Z.-S. Li, “Multi-label learning with label-specific features via weighting and label entropy guided clustering ensemble,” Neurocomputing, vol. 419, pp. 59–69, 2021. doi: 10.1016/j.neucom.2020.07.107
    J. Huang, G. Li, S. Wang, Z. Xue, and Q. Huang, “Multi-label classification by exploiting local positive and negative pairwise label correlation,” Neurocomputing, vol. 257, pp. 164–174, 2017. doi: 10.1016/j.neucom.2016.12.073
    C.-C. Chang and C.-J. Lin, “LIBSVM: A library for support vector machines,” ACM Trans. Intelligent Systems and Technology, vol. 2, no. 3, p. 27, 2011.
    J. Demšar, “Statistical comparisons of classifiers over multiple data sets,” J. Machine Learning Research, vol. 7, pp. 1–30, 2006.
    O. J. Dunn, “Multiple comparisons among means,” J. American Statistical Association, vol. 56, no. 293, pp. 52–64, 1961. doi: 10.1080/01621459.1961.10482090


    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(3)  / Tables(9)

    Article Metrics

    Article views (385) PDF downloads(40) Cited by()


    DownLoad:  Full-Size Img  PowerPoint