data.table dplyr hashTagR ggplot2 knitr kableExtra lubridate
TRUE TRUE TRUE TRUE TRUE TRUE TRUE
readr rtweet tidytext wordcloud
TRUE TRUE TRUE TRUE savingSessions: tweet analysis
1 No longer updated
See the integrated electricity demand and twitter report instead.
2 Background
UK demand response experiments by NG-ESO and retailers such as @OctopusEnergy
Attempt to do some analysis of #savingSession(s) tweets.
Inspired by https://docs.ropensci.org/rtweet/
Last run at: 2023-02-16 11:37:50
3 Setup
Part of https://github.com/dataknut/savingSessions
Makes use of https://github.com/dataknut/hashTagR, a DIY wrapper for the rtweet rstats package.
Grab the most recent set of tweets that mention #savingSession OR #savingSessions OR #savingsession using the rtweet::search_tweet() function and merge with any we may already have downloaded.
Should we also try to get all replies to @savingSessions?
Note that tweets do not seem to be available after ~ 14 days via the API used by rtweet. Best to keep refreshing the data every week…
That produced a data file of 5888 tweets.
We do NOT store the tweets in the repo for both ethical and practical reasons…
Note also that we may not be collecting the complete dataset of hashtagged tweets due to the intricacies of the twitter API.
4 Analysis
4.1 Overall tweet time line
Figure 1 shows the timing of tweets by hour.
Figure 2 shows cumulative tweets by hour.
There are interesting variations in the ‘attention’ each session gets - if we can assume that the number of tweets is some indicator of ‘attention’.
4.2 Overall content analysis
Let’s try a word cloud.
Inspiration here: https://towardsdatascience.com/create-a-word-cloud-with-r-bde3e7422e8a
Make a word cloud for all words in all tweets.
These may not render the word ‘savingsession’ as it will be in all tweets due to the twitter search pattern used.
We need to remove common words (to, the, and, a, for, etc). These are called ‘stop words’.
What happens if we do that?
Not especially informative… Perhaps we should try to extract the ‘sentiment’ of the words.
4.3 Sentiment analysis (all tweets)
Inspired by https://www.tidytextmining.com/sentiment.html
Take those cleaned words and sentiment them!
In each case we show the number of negative and positive codings for the unique words (which will add up to the Number of unique words by sentiment) and then the total frequency of words that are negative or positive (which will add up to the total number of words).
Got it?
The first word cloud shows names that have negative sentiment (according to tidytext::get_sentiments("bing")). Remember the size of the words is relative to the count of all negative words.
[1] 648
negative positive
353 295 [1] 5080# A tibble: 2 × 2
sentiment freq
<chr> <int>
1 negative 1505
2 positive 3575
The second wordcloud shows words with positive sentiments. Remember the size of the words is relative to the count of all positive words.
Joining with `by = join_by(word)`
The final plot shows trends in negative and positive sentiment over time.
4.4 Session by session sentiment analysis
4.4.1 Session 1 - (Tue 15 Nov 2022)
Repeat these negative/positive word clouds for Session 1 which was on Tue 15 Nov 2022.
These are just the tweets for the day of the event and the day after…
Positive words…
Joining with `by = join_by(word)`
Joining with `by = join_by(word)`
Negative words…
Joining with `by = join_by(word)`
Joining with `by = join_by(word)`
Total word and unique word counts…
| Var1 | Freq |
|---|---|
| negative | 87 |
| positive | 104 |
| sentiment | freq |
|---|---|
| negative | 203 |
| positive | 349 |
4.4.2 Session 2 - (Tue 22 Nov 2022)
Repeat these negative/positive word clouds for Session 2 which was on Tue 22 Nov 2022.
These are just the tweets for the day of the event and the day after…
Positive words…
Joining with `by = join_by(word)`
Joining with `by = join_by(word)`
Negative words…
Joining with `by = join_by(word)`
Joining with `by = join_by(word)`
Total word and unique word counts…
| Var1 | Freq |
|---|---|
| negative | 107 |
| positive | 106 |
| sentiment | freq |
|---|---|
| negative | 197 |
| positive | 410 |
4.4.3 Session 3 - (Wed 30 Nov 2022)
Repeat these negative/positive word clouds for Session 3 which was on Wed 30 Nov 2022.
These are just the tweets for the day of the event and the day after…
Positive words…
Joining with `by = join_by(word)`
Joining with `by = join_by(word)`
Negative words…
Joining with `by = join_by(word)`
Joining with `by = join_by(word)`
Total word and unique word counts…
| Var1 | Freq |
|---|---|
| negative | 77 |
| positive | 103 |
| sentiment | freq |
|---|---|
| negative | 206 |
| positive | 490 |
4.4.4 Session 4 - (Thu 01 Dec 2022)
Repeat these negative/positive word clouds for Session 4 which was on Thu 01 Dec 2022.
These are just the tweets for the day of the event and the day after…
Positive words…
Joining with `by = join_by(word)`
Joining with `by = join_by(word)`
Negative words…
Joining with `by = join_by(word)`
Joining with `by = join_by(word)`
Total word and unique word counts…
| Var1 | Freq |
|---|---|
| negative | 40 |
| positive | 62 |
| sentiment | freq |
|---|---|
| negative | 83 |
| positive | 189 |
4.4.5 Session 5 - (Mon 12 Dec 2022)
Repeat these negative/positive word clouds for Session 5 which was on Mon 12 Dec 2022.
These are just the tweets for the day of the event and the day after…
Positive words…
Joining with `by = join_by(word)`
Joining with `by = join_by(word)`
Negative words…
Joining with `by = join_by(word)`
Joining with `by = join_by(word)`
Total word and unique word counts…
| Var1 | Freq |
|---|---|
| negative | 41 |
| positive | 73 |
| sentiment | freq |
|---|---|
| negative | 138 |
| positive | 362 |
4.4.6 Session 6 - (Thu 19 Jan 2023)
Repeat these negative/positive word clouds for Session 6 which was on Thu 19 Jan 2023.
These are just the tweets for the day of the event and the day after…
Positive words…
Joining with `by = join_by(word)`
Joining with `by = join_by(word)`
Negative words…
Joining with `by = join_by(word)`
Joining with `by = join_by(word)`
Total word and unique word counts…
| Var1 | Freq |
|---|---|
| negative | 22 |
| positive | 40 |
| sentiment | freq |
|---|---|
| negative | 37 |
| positive | 85 |
4.4.7 Session 7 - (Mon 23 Jan 2023)
Repeat these negative/positive word clouds for Session 7 which was on Mon 23 Jan 2023.
These are just the tweets for the day of the event and the day after…
Positive words…
Joining with `by = join_by(word)`
Joining with `by = join_by(word)`
Negative words…
Joining with `by = join_by(word)`
Joining with `by = join_by(word)`
Total word and unique word counts…
| Var1 | Freq |
|---|---|
| negative | 92 |
| positive | 94 |
| sentiment | freq |
|---|---|
| negative | 245 |
| positive | 417 |
4.4.8 Session 8 - (Tue 24 Jan 2023)
Repeat these negative/positive word clouds for Session 8 which was on Tue 24 Jan 2023.
These are just the tweets for the day of the event and the day after…
Positive words…
Joining with `by = join_by(word)`
Joining with `by = join_by(word)`
Negative words…
Joining with `by = join_by(word)`
Joining with `by = join_by(word)`
Total word and unique word counts…
| Var1 | Freq |
|---|---|
| negative | 82 |
| positive | 76 |
| sentiment | freq |
|---|---|
| negative | 224 |
| positive | 324 |