Inspite of the level of popularity of virtual musicians, most of them can not participate in together with human musicians subsequent their tempo or make their individual behaviors without the need of the support of human people. The authors of a current paper established a virtual violinist possessing these attributes.

Picture credit: 刘睿忱 by means of Wikimedia (CC BY-SA four.)

It can observe tunes and adapt to the human pianist’s tempo different with time and with efficiency, building the two voices harmonized. The virtual musician’s physique movements are generated straight from the tunes. The motion generator is properly trained on a tunes video dataset of violin efficiency and a pose sequence synchronized with live efficiency is generated.

These features imply that the human musician can practice, rehearse, and perform tunes with the virtual musician like with a real human, by subsequent the tunes articles. The proposed method has successfully carried out in a ticket-marketing concert, where by a movement from Beethoven’s Spring Sonata was played. 

Digital musicians have grow to be a amazing phenomenon in the contemporary multimedia arts. On the other hand, most of the virtual musicians presently have not been endowed with talents to make their individual behaviors, or to perform tunes with human musicians. In this paper, we to begin with make a virtual violinist, who can collaborate with a human pianist to perform chamber tunes routinely without the need of any intervention. The method incorporates the approaches from a variety of fields, together with real-time tunes monitoring, pose estimation, and physique movement era. In our method, the virtual musician’s actions is generated based on the offered tunes audio alone, and these types of a method success in a minimal-charge, effective and scalable way to create human and virtual musicians’ co-efficiency. The proposed method has been validated in community concert events. Objective high-quality evaluation strategies and feasible methods to systematically strengthen the method are also reviewed.

Link: https://arxiv.org/abs/2009.07816