What Trends Are On The Horizon in Music?
In a recent project, I examined data from the Spotify API to understand where the music scene is headed over the next five years. I used classification analysis to identify which features would predict popularity and which features could be considered niche. I figured it would be nice to not only build something that could be used by music executives but also independent artists. A music executive, of course, will want to know which features to invest in when funding new artists and they will want to know, in particular, which features will predict popularity however an independent artist may be looking to see which features will be underused in the coming years and, perhaps, may take advantage of this information to stand out.
Once I identified my features with classification, I engaged in a time series analysis to determine how prevalent each of these features would be over the next five years.
A quick note: It may feel, at times, like important technical details are overlooked. If you are looking to gain more information about my work, please visit the project’s readme at the GitHub Repository here. If you would like more information on Classification Analysis Models, the best place to go would be to a separate project I worked on that was solely dedicated to classification.
Classification
For this part of my project, I built a series of around 8 models trained on all of the features. After this, I reevaluated the top 7 features to confirm the four best features for popularity prediction.
The models I built were primarily decision tree & random forest classification models. I began each model’s process with an untuned model (meaning I did not change any of the default parameters) and then proceeded to manually tune and/or grid-search to find the best parameters for the best results.
One of the most important factors when examining the models’ results for validation was the AUC. This feature lets you know how well your model can detect the difference between classes. This is especially important when you have a class imbalance in your data (which I had: 22% of the 170,000 were deemed popular and 78% were unpopular).
The following three images are of the AUCs from three of my many models. The best AUC comes from a curve that is closest to the top left corner of your graph.
From the Grid-Searched Random Forest model, by far our best, we were able to identify the four most popular and most niche features. The following two bar graphs reflect a model trained on all our features and then on our top 7 features.
Our Four Most Popular Features:
- Loudness: The overall loudness of a track in decibels (dB). Measured between -60 and 0 because it is digital sound. For more information on negative decibels, read this great article.
- Acousticness: A confidence measure from 0.0 to 1.0 of whether the track is acoustic. 1.0 represents high confidence the track is acoustic.
- Energy: A measure from 0.0 to 1.0 and represents a perceptual measure of intensity and activity. Typically, energetic tracks feel fast, loud, and noisy. For example, death metal has high energy, while a Bach prelude scores low on the scale.
- Valence: A measure from 0.0 to 1.0 describing the musical positiveness conveyed by a track. Tracks with high valence sound more positive (e.g. happy, cheerful, euphoric), while tracks with low valence sound more negative (e.g. sad, depressed, angry).
Our Four Most Niche Features:
- Mode: Indicates the modality (major or minor) of a track, the type of scale from which its melodic content is derived. Major is represented by 1 and minor is 0.
- Tempo: The overall estimated tempo of a track in beats per minute (BPM). In musical terminology, tempo is the speed or pace of a given piece and derives directly from the average beat duration.
- Speechiness: Detects the presence of spoken words in a track. The more exclusively speech-like the recording (e.g. talk show, audio book, poetry), the closer to 1.0 the attribute value.
- Liveness: Detects the presence of an audience in the recording. Higher liveness values represent an increased probability that the track was performed live. A value above 0.8 provides strong likelihood that the track is live.
Note: Leave a comment if you would like an explanation of why I left out the feature “key” :)
Time Series Analysis
After completing classification analysis, I moved on to time series analysis. The goal, here, was to forecast the prevalence of our features over the course of the next five years.
Before heading into time series analysis for each individual, I wanted to visualize a preliminary time series for all of our features together. One visualization will have our popular features and the other will have our niche features.
Take a second and think about what you see. It seems that acousticness was fairly popular up until around the 50s where it saw a steep decline until 1980 where it has since plateaued, more or less. Valence (cheerfulness) saw a rise between 1950 and 1980 and has since plunged up until around two years ago where it is on the rise again! Energy and loudness have been on a solid ascent since the 50s and never really saw a dip until the 10s although we are seeing a rise in both of those, once again.
Take a look at these niche features. Speechiness took a dive in 1950 and hasn’t really recovered. Tempo rose until 1980, dipped a bit and has been up and down since 2000.
After these visualizations, I created a separate time series for each feature (which is to say that I separated each feature into its own dataframe with the year it was released as the index). From there, I built a separate ARIMA time series model for each feature.
The best performing model, by far, was the model I built for loudness. The Root Mean Squared Error (the value I used for validation) was incredibly low for both the train and test groups, as modeled below:
After receiving forecasts from all 8 models, the values I received were as follows:
For the Popular Features:
- Loudness: 44.6% increase over 5 years
- Energy: 31.06% increase over 5 years
- Valence: 2.24% increase over 5 years
- Acousticness: 1.52% increase over 5 years
For the Niche Features:
- Tempo: 37.2% increase over 5 years
- Liveness: 2.95% increase over 5 years
- Mode: 2.23% increase over 5 years (can be read as: Scales will stay minor)
- Speechiness: 1.98% increase over 5 years
Conclusion for Popular Features:
The prevalence of Loudness & Energy will grow the most of the 4 features that most likely predict popularity.
If you are an executive looking to emphasize certain features in the artists sponsored by your label, you may want to pursue these features if following trends is your thing however if you are looking to break the mold, investing in acoustic artists may be ideal as the prevalence of acousticness is quite low right now.
Conclusion for Niche Features:
Include Speechiness & a Major Scale to stand out.
If you are an artist looking to tap into features that don’t necessarily correlate with popularity (think Top 40 Radio Hits) then this information may be pertinent in your decision-making for the direction of your music.
Tempo is on trend to increase dramatically more than the other features (i.e.: Songs with higher tempo will continue to be prevalent.) however speechiness will be rarely used and does not necessarily predict popularity. If you’re willing to break the mold in your cohort and your goal isn’t necessarily world fame, this may be a feature to include in your song. If you’re the type of person who doesn’t care about popularity, you may not necessarily care about what I have to say here however if you are an artist who includes speechiness in their music, a la Johnny Cash, rest assured that there will not be many artists like you out there.
There will be a separate blogpost solely dedicated to LSTMs.
If you liked this, please check out the rest of my work!