June 24, 2021 From rOpenSci (https://deploy-preview-488--ropensci.netlify.app/blog/2021/06/24/news-meta/). Except where otherwise noted, content on this site is licensed under the CC-BY license.
How to keep up with rOpenSci? We agree that we’re doing so much good work that it’s hard. 😉 More seriously, we’ve been curating and sharing a news digest with our community for years because we believe it to be useful. Over time its structure and infrastructure have evolved. In this post we’ll share how we currently prepare content for the newsletter and send it to subscribers' mailboxes, as automatically as possible.
rOpenSci has had a newsletter since 2014!1 For the longest time the newsletter has been living at a Jekyll website at a subdomain; whose entries were also sent as email to subscribers. It was published mostly once every two weeks.2 The preparation of each post was helped with a Makefile, and some manual manipulation and prose writing.
Later, the newsletter got complemented by posts on our main blog called “two months in two minutes” that were digests of the digests.3
For a few months now we have settled on:
Find the June newsletter. The newsletter structure is (🏭 for automatically generated content, ✍️ for manually curated sections):
Having the newsletter available as entries online somewhere is very handy for linking to them from e.g. Twitter. Furthermore, we really like our website (redesigned last summer ✨) so it felt logical to have the newsletter live in it, especially as it meant one could hop from the newsletter to other website pages much more easily. Having the newsletter issues as blog posts was the most natural way to integrate them here, but we’ve also:
Now, how do we go from “information out there” to a formatted blog post? We use an R Markdown document that needs the date of the latest newsletter as parameter. That document is stored as an Hugo archetype so we can create a newsletter via the blogdown New Post addin in RStudio. It’s filled with chunks producing text.
Compared to the previous way we created newsletters, our new method relies more heavily on data accessed directly from GitHub:
DESCRIPTION when they’re listed there, otherwise we manually add a snippet of code to the R Markdown document to add their names to the package presentation.The use cases data comes from our forum via Discourse API whose client discgolf is maintained by rOpenSci Scott Chamberlain. The same JSON data feeds our use cases page.
A random tip is that we use english::english() to transcribe numbers to English.
E.g. as.character(english::english(2021)) gives “two thousand and twenty-one”.
An alternative (that we had not heard of when creating the R Markdown document) is xfun::numbers_to_words().
We still have some progress to make e.g. instead of using a DESCRIPTION file to store dependencies, we should probably switch to using renv.
Let’s see, we may wait for something to break before making the change!
The newsletter has a few manually curated sections. To populate this, we use information from the website (comm calls), our reading (was an interesting answer or question sent to the R-package-devel mailing list?), etc. To not forget ideas or to bring them to the attention of the newsletter curator we store them as issues in a GitHub repository.
Furthermore, the manually curated sections are reviewed by at least one other person. The posts don’t get a full blown review but this helps strengthen them.
We used to send our newsletter via MailChimp, however we’ve been hearing problematic things about the company itself so asked around for recommendations (thanks a ton to everyone who chimed in!) and settled on SendGrid. SendGrid offers both email and marketing services. We were already using email services for the CRAN checks API and our forum. We now also use SendGrid marketing services, for sending our newsletter.4
How do we go about that? We extract the HTML corresponding to the post from our full content RSS feed (set up for R Bloggers). We use that feed as all relative links (except for images) have been made absolute at that point. We copy-paste it into a text editor and make a few tweaks:
These tweaks are listed in our newsletter checklist but we might transform them to code using xml2 soon-ish.
We then copy this HTML as a code block in SendGrid visual editor. Regarding the email appearance, we added some blocks (e.g. the unsubscribe one at the bottom), tweaked a few styling rules (e.g. the color of text, background, links, to match our website). To send a new newsletter we duplicate a previous one. Folks more experienced with SendGrid might create a “template” for that, presumably.
We make sure the subject and preheader (what MailChimp calls preview, the email bit recipients might see near the subject in their inbox) are updated, then send a few test emails, and then more or less nervously send the newsletter to its more than 2,000 subscribers.
Voilà, at that point all that remains is tweeting about the newsletter, and hoping it gets read at least a bit. A problem we encountered with our first SendGrid campaign is the rendering of the font on some devices, which we will investigate by next time.
Note that SendGrid sign-up forms offer much less flexibility than MailChimp similar features, so we had to add an iframe to our website to incorporate sign-up, where we used to have a more integrated form whose JS code pinged a dedicated MailChimp URL.
In this post we presented our workflow for curating and sending our monthly newsletter. Our newsletter is published as a monthly post on our blog, and sent as an email to subscribers via SendGrid. We prepare its content with a mix of automatically generated and manually curated sections, followed by a review for the latter. The automatic generation of some content depends on upstream manual and automatic work for gathering data (e.g. our packages registry, citations monitoring, encouraging users to report use cases). We hope the newsletter helps our community stay informed of our work. If you have any suggestion for our newsletter, don’t hesitate to leave a comment below! And to subscribe, head over to the newsletter page!
Reading that first issue is amusing as it announces the beginning of comm calls, that are now a well established part of rOpenSci activities and have a very polished flow. ↩︎
That is why it was called rOpenSci biweekly, although not everyone agrees on what frequency biweekly means (twice a week or once every two weeks?). ↩︎
The newsletter section about blog posts mentioned these digests. ↩︎
Transferring contacts merely means exporting subscribed contacts out of MailChimp and then importing them into SendGrid. As we assume most people unsubscribe from a very recent newsletter, we should not miss unsubscribe events. Note that to add contacts by CSV to SendGrid make sure to remove any space in the column headers (MailChimp CSV export is not usable for SendGrid without these tweaks). ↩︎
We do track opens, via a 1px square image, which you can turn off by not letting your email service load external images by default. ↩︎
