| 10 |
Listening to topics of interestConceptBeam: Technology for separating signals by meaning 
|
|---|
We are researching new ways to mathematically express the “meaning” intrinsically communicated by sounds, images, and text. Here, we introduce novel technology for extracting targeted speech from an audio signal containing several speakers talking about several topics all at once by filtering for a specific conceptual “meaning” embodied in images or sounds. This is a new type of sound source separation technology. Historically, research in this field has focused on using the physical properties of the signal itself (direction of sound arrival, statistical independence of source signals, speaker characteristics, etc.) to extract the desired signal. In contrast, our method, named ConceptBeam, integrates semantic information directly into the signal processing for source separation, enabling content-based semantic extraction of desired signals. In this age of information overload, we aim to realize a society in which information of interest, regardless of format, can be quickly and accurately extracted from vast streams of data by combining semantic processing with traditional signal processing and pattern recognition technologies.
[1] Y. Ohishi, M. Delcroix, T. Ochiai, S. Araki, D. Takeuchi, D. Niizumi, A. Kimura, N. Harada, K. Kashino, “ConceptBeam: Concept Driven Target Speech Extraction,” in Proc. ACM Multimedia, pp. 4252-4260, 2022.
[2] M. Delcroix, K. Zmolikova, K. Kinoshita, S. Araki, A. Ogawa, T. Nakatani, “SpeakerBeam: A New Deep Learning Technology for Extracting Speech of a Target Speaker Based on the Speaker’s Voice Characteristics,” NTT Technical Review, Vol. 16, No. 11, pp. 20-24, 2018．
Kunio Kashino
Biomedical Informatics Research Group, Media Information Laboratory