1 No longer updated

See the integrated electricity demand and twitter report instead.

2 Background

UK demand response experiments by NG-ESO and retailers

Attempt to do some analysis of #DemandFlexibilityService (s) tweets.

Inspired by https://docs.ropensci.org/rtweet/

Last run at: 2023-01-25 13:52:56

3 Setup

Part of https://github.com/dataknut/savingSessions

Makes use of https://github.com/dataknut/hashTagR, a DIY wrapper for the rtweet rstats package.

data.table      dplyr   hashTagR    ggplot2      knitr kableExtra  lubridate 
      TRUE       TRUE       TRUE       TRUE       TRUE       TRUE       TRUE 
     readr     rtweet   tidytext  wordcloud 
      TRUE       TRUE       TRUE       TRUE

Grab the most recent set of tweets that mention #DemandFlexibilityService using the rtweet::search_tweet() function and merge with any we may already have downloaded.

Should we also try to get all replies to @savingSessions?

Note that tweets do not seem to be available after ~ 14 days via the API used by rtweet. Best to keep refreshing the data every week…

[1] "Found 36 files matching *.csv in ~/Dropbox/data/twitter/demandFlexibilityService/"

That produced a data file of 240 tweets.

We do NOT store the tweets in the repo for both ethical and practical reasons…

Note also that we may not be collecting the complete dataset of hashtagged tweets due to the intricacies of the twitter API.

4 Analysis

Figure 1 shows the timing of tweets by hour.

Figure 2 shows cumulative tweets by hour.

Figure 2: Cumulative number of tweets over time

Let’s try a word cloud.

Inspiration here: https://towardsdatascience.com/create-a-word-cloud-with-r-bde3e7422e8a

Make a word cloud for all words in all tweets.

These may not render the word ‘savingsession’ as it will be in all tweets due to the twitter search pattern used.

We need to remove common words (to, the, and, a, for, etc). These are called ‘stop words’.

What happens if we do that?

Not especially informative… Perhaps we should try to extract the ‘sentiment’ of the words.

Inspired by https://www.tidytextmining.com/sentiment.html

Take those cleaned words and sentiment them!

In each case we show the number of negative and positive codings for the unique words (which will add up to the Number of unique words by sentiment) and then the total frequency of words that are negative or positive (which will add up to the total number of words).

Got it?

The first word cloud shows names that have negative sentiment (according to tidytext::get_sentiments("bing")). Remember the size of the words is relative to the count of all negative words.

[1] 56


negative positive 
      23       33

[1] 187

# A tibble: 2 × 2
  sentiment  freq
  <chr>     <int>
1 negative     78
2 positive    109

The second wordcloud shows words with positive sentiments. Remember the size of the words is relative to the count of all positive words.

Joining, by = "word"

The final plot shows trends in negative and positive sentiment over time.

2023-01-23 17:00:00 to
2023-01-23 18:00:00

Repeat these negative/positive word clouds but just for the first session.

These are just the tweets for the day of the event and the day after…

Positive words…

Joining, by = "word"
Joining, by = "word"

Number of unique words by sentiment
Var1	Freq
negative	23
positive	32

Number of words
sentiment	freq
negative	68
positive	108

Negative words…

Joining, by = "word"
Joining, by = "word"

2023-01-24 16:30:00 to
2023-01-24 18:00:00

Repeat these negative/positive word clouds but for the second session.

These are just the tweets for the day of the event and the day after…

Positive words…

No data yet, try later!

Negative words…

No data yet, try later!