A journal of IEEE and CAA , publishes high-quality papers in English on original theoretical/experimental research and development in all areas of automation
Advanced Search
Volume 9 Issue 7
Jul.  2022

IEEE/CAA Journal of Automatica Sinica

  • JCR Impact Factor: 15.3, Top 1 (SCI Q1)
    CiteScore: 23.5, Top 2% (Q1)
    Google Scholar h5-index: 77， TOP 5
Turn off MathJax
Article Contents
Article Navigation >  IEEE/CAA Journal of Automatica Sinica  > 2022  >  9(7): 1248-1261
Y.-B. Wang, J.-Y. Hang, and M.-L. Zhang, “Stable label-specific features generation for multi-label learning via mixture-based clustering ensemble,” IEEE/CAA J. Autom. Sinica, vol. 9, no. 7, pp. 1248–1261, Jul. 2022. doi: 10.1109/JAS.2022.105518
Citation: Y.-B. Wang, J.-Y. Hang, and M.-L. Zhang, “Stable label-specific features generation for multi-label learning via mixture-based clustering ensemble,” IEEE/CAA J. Autom. Sinica, vol. 9, no. 7, pp. 1248–1261, Jul. 2022. doi: 10.1109/JAS.2022.105518
shu

Stable Label-Specific Features Generation for Multi-Label Learning via Mixture-Based Clustering Ensemble

doi: 10.1109/JAS.2022.105518

  • School of Computer Science and Engineering, Southeast University, Nanjing 210096, China

Funds:  This work was supported by the National Science Foundation of China (62176055) and the China University S&T Innovation Plan Guided by the Ministry of Education
More Information
    Abstract
  • Multi-label learning deals with objects associated with multiple class labels, and aims to induce a predictive model which can assign a set of relevant class labels for an unseen instance. Since each class might possess its own characteristics, the strategy of extracting label-specific features has been widely employed to improve the discrimination process in multi-label learning, where the predictive model is induced based on tailored features specific to each class label instead of the identical instance representations. As a representative approach, LIFT generates label-specific features by conducting clustering analysis. However, its performance may be degraded due to the inherent instability of the single clustering algorithm. To improve this, a novel multi-label learning approach named SENCE (stable label-Specific features gENeration for multi-label learning via mixture-based Clustering Ensemble) is proposed, which stabilizes the generation process of label-specific features via clustering ensemble techniques. Specifically, more stable clustering results are obtained by firstly augmenting the original instance repre-sentation with cluster assignments from base clusters and then fitting a mixture model via the expectation-maximization (EM) algorithm. Extensive experiments on eighteen benchmark data sets show that SENCE performs better than LIFT and other well-established multi-label learning algorithms.

     

    • FullText(HTML)
  • loading
    • References(45)
  • [1]
    M.-L. Zhang and Z.-H. Zhou, “A review on multi-label learning algorithms,” IEEE Trans. Knowledge and Data Engineering, vol. 26, no. 8, pp. 1819–1837, 2014. doi: 10.1109/TKDE.2013.39
    [2]
    A. McCallum, “Multi-label text classification with a mixture model trained by EM,” in Proc. Working Notes AAAI’99 Workshop Text Learning, Orlando, FL, 1999.
    [3]
    M. R. Boutell, J. Luo, X. Shen, and C. M. Brown, “Learning multi-label scene classification,” Pattern Recognition, vol. 37, no. 9, pp. 1757–1771, 2004. doi: 10.1016/j.patcog.2004.03.009
    [4]
    H. Kazawa, T. Izumitani, H. Taira, and E. Maeda, “Maximal margin labeling for multi-topic text categorization,” in Proc. 17th Int. Conf. Neural Information Processing Systems, Vancouver, Canada, 2004, pp. 649–656.
    [5]
    Z. Barutcuoglu, R. E. Schapire, and O. G. Troyanskaya, “Hierarchical multi-label prediction of gene function,” Bioinformatics, vol. 22, no. 7, pp. 830–836, 2006. doi: 10.1093/bioinformatics/btk048
    [6]
    M.-L. Zhang and L. Wu, “LIFT: Multi-label learning with labelspecific features,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 37, no. 1, pp. 107–120, 2014.
    [7]
    A. Topchy, J. A. K, and W. Punch, “A mixture model for clustering ensembles,” in Proc. SIAM Int. Conf. Data Mining, Florida, USA, 2004, pp. 379–390.
    [8]
    W. Zhan and M.-L. Zhang, “Multi-label learning with label-specific features via clustering ensemble,” in Proc. IEEE Int. Conf. Data Science and Advanced Analytics, Tokyo, Japan, 2017, pp. 129–136.
    [9]
    Z.-H. Zhou and W. Tang, “Clusterer ensemble,” Knowledge-Based Systems, vol. 19, no. 1, pp. 77–83, 2006. doi: 10.1016/j.knosys.2005.11.003
    [10]
    H. Ayad and M. Kamel, “Finding natural clusters using multi-clusterer combiner based on shared nearest neighbors,” in Proc. 4th Int. Workshop Multiple Classifier Systems, Surrey, UK, 2003, pp. 166–175.
    [11]
    E. Gibaja and S. Ventura, “A tutorial on multilabel learning,” ACM Computing Surveys, vol. 47, no. 3, pp. 1–38, 2015.
    [12]
    M.-L. Zhang, Y.-K. Li, Y.-Y. Liu, and X. Geng, “Binary relevance for multi-label learning: An overview,” Frontiers of Computer Science, vol. 12, no. 2, pp. 191–202, 2018. doi: 10.1007/s11704-017-7031-7
    [13]
    C. Brinker, E. Loza Mencía, and J. Fürnkranz, “Graded multilabel classification by pairwise comparisons,” in Proc. 14th IEEE Int. Conf. Data Mining, Shenzhen, China, 2014, pp. 731–736.
    [14]
    J. Fürnkranz, E. Hüllermeier, E. Loza Mencía, and K. Brinker, “Multilabel classification via calibrated label ranking,” Machine Learning, vol. 73, no. 2, pp. 133–153, 2008. doi: 10.1007/s10994-008-5064-8
    [15]
    J. Read, B. Pfahringer, G. Holmes, and E. Frank, “Classifier chains for multi-label classification,” Machine Learning, vol. 85, no. 3, pp. 333–359, 2011. doi: 10.1007/s10994-011-5256-5
    [16]
    G. Tsoumakas, I. Katakis, and I. Vlahavas, “Random k-labelsets for multi-label classification,” IEEE Trans. Knowledge and Data Engineering, vol. 23, no. 7, pp. 1079–1089, 2011. doi: 10.1109/TKDE.2010.164
    [17]
    M. Huang, F. Zhuang, X. Zhang, X. Ao, Z. Niu, M.-L. Zhang, and Q. He, “Supervised representation learning for multi-label classification,” Machine Learning, vol. 108, no. 5, pp. 747–763, 2019. doi: 10.1007/s10994-019-05783-5
    [18]
    L. Sun, S. Ji, and J. Ye, Multi-Label Dimensionality Reduction. Boca Ration, FL: Chapman and Hall/CRC, 2013.
    [19]
    C. Yan, X. Chang, M. Luo, Q. Zheng, X. Zhang, Z. Li, and F. Nie, “Selfweighted robust lda for multiclass classification with edge classes,” ACM Trans. Intelligent Systems and Technology, vol. 12, no. 1, pp. 1–19, 2020.
    [20]
    X. Chang, F. Nie, S. Wang, Y. Yang, X. Zhou, and C. Zhang, “Compound rank-k projections for bilinear analysis,” IEEE Trans. Neural Networks and Learning Systems, vol. 27, no. 7, pp. 1502–1513, 2015.
    [21]
    R. B. Pereira, A. Plastino, B. Zadrozny, and L. H. C. Merschmann, “Categorizing feature selection methods for multi-label classification,” Artificial Intelligence Review, vol. 49, no. 1, pp. 57–78, 2018. doi: 10.1007/s10462-016-9516-4
    [22]
    Z. Cai and W. Zhu, “Feature selection for multi-label classification using neighborhood preservation,” IEEE/CAA J. Autom. Sinica, vol. 5, no. 1, pp. 320–330, 2017.
    [23]
    C. Yan, Q. Zheng, X. Chang, M. Luo, C.-H. Yeh, and A. G. Hauptman, “Semantics-preserving graph propagation for zero-shot object detection,” IEEE Trans. Image Processing, vol. 29, pp. 8163–8176, 2020. doi: 10.1109/TIP.2020.3011807
    [24]
    Y. Chen, X. Yang, J. Li, P. Wang, and Y. Qian, “Fusing attribute reduction accelerators,” Information Sciences, vol. 587, pp. 354–370, 2022. doi: 10.1016/j.ins.2021.12.047
    [25]
    S. Canuto, M. A. Gonçalves, and F. Benevenuto, “Exploiting new sentiment-based meta-level features for effective sentiment analysis,” in Proc. 9th ACM Int. Conf. Web Search and Data Mining, San Francisco, CA, 2016, pp. 53–62.
    [26]
    Y. Yang and S. Gopal, “Multilabel classification with meta-level features in a learning-to-rank framework,” Machine Learning, vol. 88, no. 1−2, pp. 47–68, 2012. doi: 10.1007/s10994-011-5270-7
    [27]
    X. Wu, Q.-G. Chen, Y. Hu, D. Wang, X. Chang, X. Wang, and M.-L. Zhang, “Multi-view multi-label learning with view-specific information extraction,” in Proc. 28th Int. Joint Conf. Artificial Intelligence, Macau, China, 2019, pp. 3884–3890.
    [28]
    C. Zhang, Z. Yu, Q. Hu, P. Zhu, X. Liu, and X. Wang, “Latent semantic aware multi-view multi-label classification,” in Proc. 32nd AAAI Conf. Artificial Intelligence, New Orleans, LA, 2018, pp. 4414–4421.
    [29]
    W. Zhan and M.-L. Zhang, “Inductive semi-supervised multi-label learning with co-training,” in Proc. 23rd ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, 2017, pp. 1305–1314.
    [30]
    J. Huang, G. Li, Q. Huang, and X. Wu, “Learning label-specific features and class-dependent labels for multi-label classification,” IEEE Trans. Knowledge and Data Engineering, vol. 28, no. 12, pp. 3309–3323, 2016. doi: 10.1109/TKDE.2016.2608339
    [31]
    W. Weng, Y. Chen, C. Chen, S. Wu, and J. Liu, “Non-sparse label specific features selection for multi-label classification,” Neuro-computing, vol. 377, pp. 85–94, 2020. doi: 10.1016/j.neucom.2019.10.016
    [32]
    X.-Y. Jia, S.-S. Zhu, and W.-W. Li, “Joint label-specific features and correlation information for multi-label learning,” J. Computer Science and Technology, vol. 35, no. 2, pp. 247–258, 2020. doi: 10.1007/s11390-020-9900-z
    [33]
    J. Zhang, C. Li, D. Cao, Y. Lin, S. Su, L. Dai, and S. Li, “Multi-label learning with label-specific features by resolving label correlations,” Knowledge-Based Systems, vol. 159, pp. 148–157, 2018. doi: 10.1016/j.knosys.2018.07.003
    [34]
    L. Sun, M. Kudo, and K. Kimura, “Multi-label classification with meta-label-specific features,” in Proc. 23rd Int. Conf. Pattern Recognition, Cancun, Mexico, 2016, pp. 1612–1617.
    [35]
    S. Xu, X. Yang, H. Yu, D.-J. Yu, J. Yang, and E. C. C. Tsang, “Multilabel learning with label-specific feature reduction,” Knowledge-Based Systems, vol. 104, pp. 52–61, 2016. doi: 10.1016/j.knosys.2016.04.012
    [36]
    W. Weng, Y. Lin, S. Wu, Y. Li, and Y. Kang, “Multi-label learning based on label-specific features and local pairwise label correlation,” Neurocomputing, vol. 273, pp. 385–394, 2018. doi: 10.1016/j.neucom.2017.07.044
    [37]
    J. Ma, H. Zhang, and T. W. S. Chow, “Multilabel classification with label-specific features and classifiers: A coarse-tuned and fine-tuned framework,” IEEE Trans. Cybernetics, vol. 51, no. 2, pp. 1028–1042, 2019.
    [38]
    Z.-B. Yu and M.-L. Zhang, “Multi-label classification with label-specific feature generation: A wrapped approach,” IEEE Trans. Pattern Analysis and Machine Intelligence, 2021.
    [39]
    M.-L. Zhang, J.-P. Fang, and Y.-B. Wang, “Bilabel-specific features for multi-label classification,” ACM Trans. Knowledge Discovery from Data, vol. 16, no. 1, pp. 1–23, 2021.
    [40]
    Z.-S. Chen and M.-L. Zhang, “Multi-label learning with regularization enriched label-specific features,” in Proc. 11th Asian Conf. Machine Learning, Nagoya, Japan, 2019, pp. 411–424.
    [41]
    C.-Y. Zhang and Z.-S. Li, “Multi-label learning with label-specific features via weighting and label entropy guided clustering ensemble,” Neurocomputing, vol. 419, pp. 59–69, 2021. doi: 10.1016/j.neucom.2020.07.107
    [42]
    J. Huang, G. Li, S. Wang, Z. Xue, and Q. Huang, “Multi-label classification by exploiting local positive and negative pairwise label correlation,” Neurocomputing, vol. 257, pp. 164–174, 2017. doi: 10.1016/j.neucom.2016.12.073
    [43]
    C.-C. Chang and C.-J. Lin, “LIBSVM: A library for support vector machines,” ACM Trans. Intelligent Systems and Technology, vol. 2, no. 3, p. 27, 2011.
    [44]
    J. Demšar, “Statistical comparisons of classifiers over multiple data sets,” J. Machine Learning Research, vol. 7, pp. 1–30, 2006.
    [45]
    O. J. Dunn, “Multiple comparisons among means,” J. American Statistical Association, vol. 56, no. 293, pp. 52–64, 1961. doi: 10.1080/01621459.1961.10482090
    • Related Articles
    • Supplements(0)

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(3)  / Tables(9)

    Get Citation
    PDF
    XML

    Article Metrics

    Article views (413) PDF downloads(40) Cited by()

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return