A journal of IEEE and CAA , publishes high-quality papers in English on original theoretical/experimental research and development in all areas of automation
Advanced Search
Volume 7 Issue 5
Sep.  2020

IEEE/CAA Journal of Automatica Sinica

  • JCR Impact Factor: 15.3, Top 1 (SCI Q1)
    CiteScore: 23.5, Top 2% (Q1)
    Google Scholar h5-index: 77， TOP 5
Turn off MathJax
Article Contents
Article Navigation >  IEEE/CAA Journal of Automatica Sinica  > 2020  >  7(5): 1344-1360
Arnab Rakshit, Amit Konar and Atulya K. Nagar, "A Hybrid Brain-Computer Interface for Closed-Loop Position Control of a Robot Arm," IEEE/CAA J. Autom. Sinica, vol. 7, no. 5, pp. 1344-1360, Sept. 2020. doi: 10.1109/JAS.2020.1003336
Citation: Arnab Rakshit, Amit Konar and Atulya K. Nagar, "A Hybrid Brain-Computer Interface for Closed-Loop Position Control of a Robot Arm," IEEE/CAA J. Autom. Sinica, vol. 7, no. 5, pp. 1344-1360, Sept. 2020. doi: 10.1109/JAS.2020.1003336
shu

A Hybrid Brain-Computer Interface for Closed-Loop Position Control of a Robot Arm

doi: 10.1109/JAS.2020.1003336
More Information
    Abstract
  • Brain-Computer interfacing (BCI) has currently added a new dimension in assistive robotics. Existing brain-computer interfaces designed for position control applications suffer from two fundamental limitations. First, most of the existing schemes employ open-loop control, and thus are unable to track positional errors, resulting in failures in taking necessary online corrective actions. There are examples of a few works dealing with closed-loop electroencephalography (EEG)-based position control. These existing closed-loop brain-induced position control schemes employ a fixed order link selection rule, which often creates a bottleneck preventing time-efficient control. Second, the existing brain-induced position controllers are designed to generate a position response like a traditional first-order system, resulting in a large steady-state error. This paper overcomes the above two limitations by keeping provisions for steady-state visual evoked potential (SSVEP) induced link-selection in an arbitrary order as required for efficient control and generating a second-order response of the position-control system with gradually diminishing overshoots/undershoots to reduce steady-state errors. Other than the above, the third innovation is to utilize motor imagery and P300 signals to design the hybrid brain-computer interfacing system for the said application with gradually diminishing error-margin using speed reversal at the zero-crossings of positional errors. Experiments undertaken reveal that the steady-state error is reduced to 0.2%. The paper also provides a thorough analysis of the stability of the closed-loop system performance using the Root Locus technique.

     

    • FullText(HTML)
  • loading
    • References(60)
  • [1]
    Z. T. Chen, Z. P. Wang, K. Wang, W. B. Yi, and H. Z. Qi, “Recognizing motor imagery between hand and forearm in the same limb in a hybrid brain computer interface paradigm: An online study,” IEEE Access, vol. 7, pp. 59631–59639, May 2019. doi: 10.1109/ACCESS.2019.2915614
    [2]
    E. W. Yin, T. Zeyl, R. Saab, T. Chau, D. W. Hu, and Z. T. Zhou, “A hybrid brain-computer interface based on the fusion of P300 and SSVEP scores,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. 23, no. 4, pp. 693–701, Jul. 2015. doi: 10.1109/TNSRE.2015.2403270
    [3]
    J. R. Wolpaw and E. W. Wolpaw, Brain-Computer Interfaces: Principles and Practice. Oxford, UK: Oxford University Press, 2012.
    [4]
    J. Zhao, W. Li, and M. F. Li, “Comparative study of SSVEP- and P300-based models for the telepresence control of humanoid robots,” PLoS One, vol. 10, no. 11, pp. e0142168, Nov. 2015. doi: 10.1371/journal.pone.0142168
    [5]
    X. Q. Mao, W. Li, C. W. Lei, J. Jin, F. Duan, and S. Chen, “A brain-robot interaction system by fusing human and machine intelligence,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. 27, no. 3, pp. 533–542, Mar. 2019. doi: 10.1109/TNSRE.2019.2897323
    [6]
    T. Bastos-Filho, A. Floriano, E. Couto, and R. J. M. Godinez-Tello, “Towards a system to command a robotic wheelchair based on independent SSVEP-BCI, ” in Smart Wheelchairs and Brain-Computer Interfaces, P. Diez, Ed. London, UK: Academic Press, 2018, pp. 369–379.
    [7]
    A. F. Salazar-Gomez, J. DelPreto, S. Gil, F. H. Guenther, and D. Rus, “Correcting robot mistakes in real time using EEG signals, ” in Proc. IEEE Int. Conf. Robotics and Automation, Singapore, 2017, pp. 6570–6577.
    [8]
    Y. Yu, Z. T. Zhou, Y. D. Liu, J. Jiang, E. W. Yin, N. N. Zhang, Z. H. Wang, Y. R. Liu, X. J. Wu, and D. W. Hu, “Self-paced operation of a wheelchair based on a hybrid brain-computer interface combining motor imagery and P300 potential,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. 25, no. 12, pp. 2516–2526, Dec. 2017. doi: 10.1109/TNSRE.2017.2766365
    [9]
    A. Lopes, J. Rodrigues, J. Perdigao, G. Pires, and U. Nunes, “A new hybrid motion planner: Applied in a brain-actuated robotic wheelchair,” IEEE Robot. Autom. Mag., vol. 23, no. 4, pp. 82–93, Dec. 2016. doi: 10.1109/MRA.2016.2605403
    [10]
    F. Arrichiello, P. Di Lillo, D. Di Vito, G. Antonelli, and S. Chiaverini, “Assistive robot operated via P300-based brain computer interface, ” in Proc. IEEE Int. Conf. Robotics and Automation, Singapore, 2017, pp. 6032–6037.
    [11]
    S. L. Sheng, P. P. Song, L. Y. Xie, Z. D. Luo, W. N. Chang, S. R. Jiang, H. Y. Yu, C. Zhu, J. T. C. Tan, and F. Duan, “Design of an SSVEP-based BCI system with visual servo module for a service robot to execute multiple tasks, ” in Proc. IEEE Int. Conf. Robotics and Automation, Singapore, 2017, pp. 2267–2272.
    [12]
    L. M. Alonso-Valerdi, R. A. Salido-Ruiz, and R. A. Ramirez-Mendoza, “Motor imagery based brain-computer interfaces: An emerging technology to rehabilitate motor deficits,” Neuropsychologia, vol. 79, pp. 354–363, Dec. 2015. doi: 10.1016/j.neuropsychologia.2015.09.012
    [13]
    S. Bhattacharyya, S. Shimoda, and M. Hayashibe, “A synergetic brain-machine interfacing paradigm for multi-DOF robot control,” IEEE Trans. Syst. Man Cybern.:Syst., vol. 46, no. 7, pp. 957–968, Jul. 2016. doi: 10.1109/TSMC.2016.2560532
    [14]
    S. Bhattacharyya, A. Konar, and D. N. Tibarewala, “Motor imagery and error related potential induced position control of a robotic arm,” IEEE/CAA J. Autom. Sinica, vol. 4, no. 4, pp. 639–650, Sept. 2017. doi: 10.1109/JAS.2017.7510616
    [15]
    Y. S. Zhang, P. Xu, T. J. Liu, J. Hu, R. Zhang, and D. Z. Yao, “Multiple frequencies sequential coding for SSVEP-based brain-computer interface,” PLoS One, vol. 7, no. 3, pp. e29519, Mar. 2012. doi: 10.1371/journal.pone.0029519
    [16]
    R. G. Muller-Putz, R. Scherer, C. Brauneis, and G. Pfurtscheller, “Steady-state visual evoked potential (SSVEP)-based communication: Impact of harmonic frequency components,” J. Neural Eng., vol. 2, no. 4, pp. 123–130, Dec. 2005. doi: 10.1088/1741-2560/2/4/008
    [17]
    J. R. Wolpaw, N. Birbaumer, D. J. McFarland, G. Pfurtscheller, and T. M. Vaughan, “Brain-computer interfaces for communication and control,” Clin. Neurophysiol., vol. 113, no. 6, pp. 767–791, 2002.
    [18]
    A. Khasnobish, A. Konar, D. N. Tibarewala, and A. K. Nagar, “Bypassing the natural visual-motor pathway to execute complex movement related tasks using interval type-2 fuzzy sets,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. 25, no. 1, pp. 91–105, Jan. 2017. doi: 10.1109/TNSRE.2016.2580580
    [19]
    E. Donchin, K. M. Spencer, and R. Wijesinghe, “The mental prosthesis: Assessing the speed of a P300-based brain-computer interface,” IEEE Trans. Rehabil. Eng., vol. 8, no. 2, pp. 174–179, Jun. 2000. doi: 10.1109/86.847808
    [20]
    T. Bhattacharjee, R. Kar, A. Konar, A. Lekova, and A. K. Nagar, “A general type-2 fuzzy set induced single trial P300 detection, ” in Proc. IEEE Int. Conf. Fuzzy Systems, Naples, Italy, 2017, pp. 1–6.
    [21]
    A. Riccio, L. Simione, F. Schettini, A. Pizzimenti, M. Inghilleri, M. O. Belardinelli, D. Mattia, and F. Cincotti, “Attention and P300-based BCI performance in people with amyotrophic lateral sclerosis,” Front. Hum. Neurosci., vol. 7, pp. 732, Nov. 2013.
    [22]
    S. Bhattacharyya, A. Konar, and D. N. Tibarewala, “Motor imagery, P300 and error-related EEG-based robot arm movement control for rehabilitation purpose,” Med. Biol. Eng. Comput., vol. 52, no. 12, pp. 1007–1017, Sept. 2014. doi: 10.1007/s11517-014-1204-4
    [23]
    S. K. Kim, E. A. Kirchner, A. Stefes, and F. Kirchner, “Intrinsic interactive reinforcement learning-using error-related potentials for real world human-robot interaction,” Sci. Rep., vol. 7, no. 1, pp. 17562, Dec. 2017. doi: 10.1038/s41598-017-17682-7
    [24]
    C. Alain, H. E. McNeely, Y. He, B. K. Christensen, and R. West, “Neurophysiological evidence of error-monitoring deficits in patients with schizophrenia,” Cereb. Cortex, vol. 12, no. 8, pp. 840–846, Aug. 2002. doi: 10.1093/cercor/12.8.840
    [25]
    J. G. Kerns, J. D. Cohen, A. W. McDonald Ⅲ, M. K. Johnson, V. A. Stenger, H. Aizenstein, and C. S. Carter, “Decreased conflict-and error-related activity in the anterior cingulate cortex in subjects with schizophrenia,” Am. J. Psychiatry, vol. 162, no. 10, pp. 1833–1839, Oct. 2005. doi: 10.1176/appi.ajp.162.10.1833
    [26]
    F. L. Colino, H. Howse, A. Norton, R. Trska, A. Pluta, S. J. C. Luehr, T. C. Handy, and O. E. Krigolson, “Older adults display diminished error processing and response in a continuous tracking task,” Psychophysiology, vol. 54, no. 11, pp. 1706–1713, Nov. 2017. doi: 10.1111/psyp.12907
    [27]
    P. E. Pailing and S. J. Segalowitz, “The error-related negativity as a state and trait measure: Motivation, personality, and ERPs in response to errors,” Psychophysiology, vol. 41, no. 1, pp. 84–95, Jan. 2004. doi: 10.1111/1469-8986.00124
    [28]
    A. Kumar, L. Gao, E. Pirogova, and Q. Fang, “A review of error-related potential-based brain-computer interfaces for motor impaired people,” IEEE Access, vol. 7, pp. 142451–142466, Sept. 2019. doi: 10.1109/ACCESS.2019.2944067
    [29]
    K. Ogata, Modern Control Engineering. 4th ed. New Jersey, USA: Prentice-Hall, 2002.
    [30]
    A. C. Davison and D. V. Hinkley, Bootstrap Methods and their Application. Cambridge, UK: Cambridge University Press, 1997.
    [31]
    F. Lotte, “A tutorial on EEG signal processing techniques for mental state recognition in brain-computer interfaces, ” in Guide to Brain-Computer Music Interfacing, E. R. Miranda and J. Castet, Eds. London, UK: Springer, 2014, pp. 133–161.
    [32]
    D. J. McFarland, L. M. McCane, S. V. David, and J. R. Wolpaw, “Spatial filter selection for EEG-based communication,” Electroencephalography and Clinical Neurophysiology, vol. 103, no. 3, pp. 386–394, 1997.
    [33]
    J. Kayser and C. E. Tenke, “Hemifield-dependent N1 and event-related theta/delta oscillations: An unbiased comparison of surface Laplacian and common EEG reference choices,” Int. J. Psychophysiol., vol. 97, no. 3, pp. 258–270, Sept. 2015. doi: 10.1016/j.ijpsycho.2014.12.011
    [34]
    W. Witkowski, M. Cortese, M. Cempini, J. Mellinger, N. Vitiello, and S. R. Soekadar, “Enhancing brain-machine interface (BMI) control of a hand exoskeleton using electrooculography (EOG),” J. Neuroeng. Rehabil., vol. 11, no. 1, pp. 165, Dec. 2014. doi: 10.1186/1743-0003-11-165
    [35]
    S. Theodoridis and K. Koutroumbas, Pattern Recognition. Amsterdam, Netherlands: Elsevier, 2003.
    [36]
    S. Das, A. Abraham, and A. Konar, “Particle swarm optimization and differential evolution algorithms: Technical analysis, applications and hybridization perspectives, ” in Advances of Computational Intelligence in Industrial Systems, Y. Liu, A. Sun, H. T. Loh, W. F. Lu, and E. P. Lim, Eds. Berlin, Heidelberg, Germany: Springer, 2008, pp. 1–38.
    [37]
    A. Khasnobish, S. Bhattacharyya, A. Konar, D. N. Tibarewala, and A. K. Nagar, “A two-fold classification for composite decision about localized arm movement from EEG by SVM and QDA techniques, ” in Proc. Int. Joint Conf. Neural Networks, San Jose, USA, 2011, pp. 1344–1351.
    [38]
    B. Blankertz, R. Tomioka, S. Lemm, M. Kawanabe, and K. R. Muller, “Optimizing spatial filters for robust EEG single-trial analysis,” IEEE Signal Process. Mag., vol. 25, no. 1, pp. 41–56, Jan. 2008. doi: 10.1109/MSP.2008.4408441
    [39]
    F. Lotte and C. T. Guan, “Regularizing common spatial patterns to improve BCI designs: Unified theory and new algorithms,” IEEE Trans. Biomed. Eng., vol. 58, no. 2, pp. 355–362, Feb. 2011. doi: 10.1109/TBME.2010.2082539
    [40]
    Q. B. Zhao and L. Q. Zhang, “Temporal and spatial features of single-trial EEG for brain-computer interface, ” Comput. Intell. Neurosci., vol. 2007, Article ID: 37695, 2017.
    [41]
    R. P. N. Rao and R. Scherer, “Statistical pattern recognition and machine learning in brain-computer interfaces, ” in Statistical Signal Processing for Neuroscience and Neurotechnology, K. G. Oweiss, Ed. London, UK: Academic Press, 2010, pp. 335–367.
    [42]
    K. K. Ang, Z. Y. Chin, C. C. Wang, C. T. Guan, and H. H. Zhang, “Filter bank common spatial pattern algorithm on BCI competition IV datasets 2a and 2b,” Front. Neurosci., vol. 6, pp. 39, Mar. 2012.
    [43]
    B. Chakraborty, L. Ghosh, and A. Konar, “Designing phase-sensitive common spatial pattern filter to improve brain-computer interfacing,” IEEE Trans. Biomed. Eng., vol. 67, no. 7, pp. 2064–2072, Jul. 2020.
    [44]
    E. Haselsteiner and G. Pfurtscheller, “Using time-dependent neural networks for EEG classification,” IEEE Trans. Rehabil. Eng., vol. 8, no. 4, pp. 457–463, Dec. 2000. doi: 10.1109/86.895948
    [45]
    A. Rakshit, S. Ghosh, A. Konar, and M. Pal, “A novel hybrid brain-computer interface for robot arm manipulation using visual evoked potential, ” in Proc. 9th Int. Conf. Advances in Pattern Recognition, Bangalore, India, 2017, pp. 1–6.
    [46]
    Y. Shamash, “Model reduction using the Routh stability criterion and the Pade approximation technique,” Int. J. Control, vol. 21, no. 3, pp. 475–484, Mar. 1975. doi: 10.1080/00207177508922004
    [47]
    World Medical Association, “World medical association declaration of Helsinki. Ethical principles for medical research involving human subjects,” Bull. World Health Organ., vol. 79, no. 4, pp. 373–374, 2001.
    [48]
    L. H. He, D. Hu, M. Wan, Y. Wen, K. M. von Deneen, and M. C. Zhou, “Common Bayesian network for classification of EEG-based multiclass motor imagery BCI,” IEEE Trans. Syst. Man Cybern.:Syst., vol. 46, no. 6, pp. 843–854, Jun. 2016. doi: 10.1109/TSMC.2015.2450680
    [49]
    G. Pfurtscheller and F. H. Lopes Da Silva, “Event-related EEG/MEG synchronization and desynchronization: Basic principles,” Clin. Neurophysiol., vol. 110, no. 11, pp. 1842–1857, Nov. 1999. doi: 10.1016/S1388-2457(99)00141-8
    [50]
    A. Schlogl, F. Lee, H. Bischof, and G. Pfurtscheller, “Characterization of four-class motor imagery EEG data for the BCI-competition 2005,” J. Neural Eng., vol. 2, no. 4, pp. L14–L22, Aug. 2005. doi: 10.1088/1741-2560/2/4/L02
    [51]
    J. T. Cleophas and A. H. Zwinderman, “One-sample continuous data (One-sample t-test, one-sample Wilcoxon signed rank test, 10 patients), ” in SPSS for Starters and 2nd Levelers. Cham, Germany: Springer, 2016, pp. 3–6
    [52]
    R. H. Riffenburgh, “Tests on ranked data, ” in Statistics in Medicine, 3rd ed. R. H. Riffenburgh, Ed. San Diego, USA: Academic Press, 2012, pp. 221–248.
    [53]
    X. Han, L. Ke, S. K. Gao, and X. R. Gao, “A novel system of SSVEP-based human-robot coordination,” J. Neural Eng., vol. 16, no. 1, pp. 016006, Feb. 2019. doi: 10.1088/1741-2552/aae1ba
    [54]
    K. Lee, D. Liu, L. Perroud, R. Chavarriaga, and J. del R. Millan, “A brain-controlled exoskeleton with cascaded event-related desynch-ronization classifiers,” Robot. Auton. Syst., vol. 90, pp. 15–23, Apr. 2017. doi: 10.1016/j.robot.2016.10.005
    [55]
    K. K. Ayten, M. N. Sahinkaya, and A. Dumlu, “Optimum trajectory generation for redundant/hyper-redundant manipulators,” IFAC-PapersOnLine, vol. 49, no. 21, pp. 493–500, Dec. 2016. doi: 10.1016/j.ifacol.2016.10.651
    [56]
    Q. Gao, L. X. Dou, A. N. Belkacem, and C. Chen, “Noninvasive electroencephalogram based control of a robotic arm for writing task using hybrid BCI system, ” BioMed Res. Int., vol. 2017, Article ID: 8316485, Jun. 2017.
    [57]
    H. Zeng, Y. X. Wang, C. C. Wu, A. G. Song, J. Liu, P. Ji, B. G. Xu, L. F. Zhu, H. J. Li, and P. C. Wen, “Closed-loop hybrid gaze brain-machine interface based robotic arm control with augmented reality feedback,” Front. Neurorobot., vol. 11, pp. 60, Oct. 2017. doi: 10.3389/fnbot.2017.00060
    [58]
    C. Chen, P. Zhou, A. N. Belkacem, L. Lu, R. Xu, X. T. Wang, W. J. Tan, Z. F. Qiao, P. H. Li, Q. Gao, and D. Shin, “Quadcopter robot control based on hybrid brain-computer interface system,” Sens. Mater., vol. 32, no. 3, pp. 991–1004, Mar. 2020.
    [59]
    C. W. Liu, Y. F. Fu, J. Yang, X. Xiong, H. W. Sun, and Z. T. Yu, “Discrimination of motor imagery patterns by electroencephalogram phase synchronization combined with frequency band energy,” IEEE/CAA J. Autom. Sinica, vol. 4, no. 3, pp. 551–557, Aug. 2017. doi: 10.1109/JAS.2016.7510121
    [60]
    C. S. Kim, J. W. Sun, D. Liu, Q. S. Wang, and S. G. Paek, “Removal of ocular artifacts using ICA and adaptive filter for motor imagery-based BCI,” IEEE/CAA J. Autom. Sinica, 2017. doi: 10.1109/JAS.2017.7510370
    • Related Articles
    • Supplements(0)

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(9)  / Tables(16)

    Get Citation
    PDF
    XML

    Article Metrics

    Article views (3669) PDF downloads(113) Cited by()

    Highlights

    • A novel scheme of Brain-comanded position-control of a robot arm in a 3-dimensional environment is proposed.
    • The proposed scheme uses SSVEP for random link selection, MI for movement initiation, and P300 to turn back the link on crossing the target positions.
    • Positional overshoot and steady-state error are reduced by an exponential decrease in speed and reversal of turning of the motors as the target position is crossed.
    • The fundamental contribution includes a non-traditional approach to computing transfer function of the position controllerand its parameter setting to attain stability by Root Locus analysis.

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return