Find Lab

In the previous post A domain-specific language for sheet music paragraph layout, we discussed how to concisely represent sheet music paragraph structure in computer. Orthogonally, we also need to express the sheet music vertical structure: instruments, staves, voices and their relationship.

We aim to do this concisely and intuitively. And this will be a meta-language which can be exported to other music languges, such as Lilypond, MusicXML and MEI.

Motivation

To avoid redundant writing, there are several repeat types in sheet music to organize paragraph layout. So when performing music, instead of simply from beginning to the end in order, one should obey musical signs to jump position. Probably we can separate music paragraph layout information from music staff content when recording a sheet music. I.e., create a new tiny domain-specific language to express sheet music structure. It's especially meaningful for computer musical lanugages, such as Lilypond and ABC Notation. Consider this: for a multiple voices music, in voice-wise form, we must record the repeat symbols in every voice, and carefully keep them consistent to avoid weird errors. And for the languages' implementation, it's troublesome to handle this kind semantic error. For users, it's covert and confused. Once we can record the music paragraph layout individually, we can simply record multiple voice staff in a 2-d table form, a measure-voice matrix, like this:

This regular form will facilitate music score clipping/slicing, which is useful for music education courseware and instrument training. And it also facilitates music-related deep learning tasks, such as OMR, AI music composition and so on.

How to express a music paragraph layout

We should design this language as intuitive as possible, make it understandable to users without any computer programming knowledges.

Let's start with a simple example:

Example 1

Code:

1

1, 2, 3, 4

The code is simply the list of measure indices.

Before more complicated examples, let's take this two-parts score as a transition:

Example 2

Code:

1

[1, 2, 3, 4], [5, 6, 7, 8]

Here brackets '[]' are used to segment measure indices into paragraphs.

For a long score, writing all serial numbers is tedious, so we adopt this brief form as a syntactic sugar:

1

[1..4], [5..8]

As a sheet music program, Lilypond is high-quality and sophisticated, but not as modern as its several competitors, such as Verovio and OSMD. For nowadays, let's consider a good sheet music engine: working with HTML5 facilitates animating features development, and mobile devices are the best choice for instruments playing assistance. Yes, serving requests online is an option, but the service is inevitably expensive and slow. Computing on the device is still the best practice.

Motivation

Imagine you have a smart piano at home, which is connected with a huge music staff library. Once, you want to play some music opus on a whim, and it's so bothering to retrieve music staff in either category menus or a long favorite list, therefore, you just play by memory directly. But the piano is so smart. Just a few seconds later, it understands what you are playing and displays the staff automatically.

Let's look at another case: you see a nice piano when hanging out downtown and you are just in a mood to play a piece of music and show off your talent to passers. When you sit and play, surprisingly, the piano displays the music information about what you are playing. When you finish, it says, well done! Your performance beat 95% players on this opus.

Imaginary scenes above are coming true. Now let's talk about some ideas for implementing such a technique.

Here, we founded this blog, and we will successively publish some novel ideas and discussions on topic of deep learning, music related algorithm and piano. Thanks for your reading, enjoy😎.