Running a Hybrid Event
with Open Source
LPC PC (James Bottomley, Mike Rapoport)
Email: jejb@hansenpartnership.com Matrix: @jejb:hansenpartnership.com
Twitter: jejb_ Mastodon: @jejb@mastodon.online
About Me
Container evangelist
Open Source Advocate
- Converting Business to Open Source
- Lead effort to create LF TAB
- Plumbers PC Founding Member
Kernel Developer
- SCSI Subsystem Maintainer
- PA-RISC architecture Maintainer
In Person Plumbers Pre 2020
Plumbers is organized as Tracks and Microconference
Tracks (refereed, kernel summit, Networking/eBPF, Toolchain) tend to be talk based
Microconferences are disccussion based
Decided in 2018 to make live streaming and recording available
Upped the A/V budget to afford this
Including the provision of cameras in the rooms
Microphones also became an issue
All participants in a discussion have to
use them for remote audience to hear
Solution borrowed from Kernel Recipes was catchboxes
Fun to use (encourage participation)
No injuries so far
The idea was to make Plumbers available
in real time to people who couldn't attend in person
Communication was only outbound: no inbound chat.
The Pandemic Years 2020—2021
Like other conferences Plumbers moved fully online
Evaluated options but chose BigBlueButton
because of interaction requirements
It's the Integrations, Stupid
Open source gives excellent pieces (BBB, indico, matrix ...)
But getting it all to work together requires integration
Jon Corbet wrote a custom front end for BBB (lpcfe) replacing greenlight
Integrated the Indico timetable into BBB (click link on timetable display to join)
Also attendees need login credentials, so used LDAP with extensible schema.
credentials were email address and confirmation number (from LF cvent).
Had to train attendees to actually use BBB
Standard issues like good Webcam/Microphone
Gave headset/lights/webcam as 2020 speaker gift
Found that raised hands didn't work well
So pioneered video unmute for interaction
in 2020 used BBB internal chat
Coupled with RocketChat for other interactions
No Integration ✓
Problems
Chat was hard to read on the live stream
Streamers didn't have credentials to rocketchat for hallway
Had to buy an enterprise licence to get features like notifications
Two chat systems (BBB room
chat and rocketchat) were disconnected
Retried with Matrix as chat in 2021
Every registered attendee got matrix id (@email:lpc.events)
But couldn't use email address with @
First day was a complete disaster
had done scale test but only 100 people turned up
On the day we had 900
And our massive 16 cpu matrix node was only using 1 cpu
After a complete overhaul of the Matrix
server, second day went well (see blog post)
Streaming (free tier) attendees could now interact by chat
provided they had their own matrix account
Guy Lunardi modified the BBB chat tab to use riot embedded
so even BBB chat was exposed to streamers
Going Hybrid in 2022
We already have the remote infrastructure, so it should be simple to reuse it in physical rooms ...
Particularly if we already have streaming A/V in the room
No, it's really not that simple ...
Have to integrate with in-room A/V
Which means training the A/V people
Held meetings with them before, but didn't anticipate the issues
Specifically A/V setup of remote attendees isn't obvious to local A/V people
Problems
Riot Embedded integration failed with newer BBB
Didn't discover until a few days before conference
First day had lots of room can't hear remote or vice versa
Took a while to get the Audio routing right
Two projectors didn't work as intended
Two cameras (and BBB handling), but A/V wasn't set up that way
A/V platform had to run BBB session that appeared on screen
A/V platform used a mux that didn't run chromium very well
Kept dropping out of conference
rejoin was manual (assuming A/V guy noticed)
need sharable room console in future
Significant number of other problems
Only had 2 people onsite who could fix them
Conclusions
Running conferences is hard
Running online conferences is harder
Because of the integration issues
Hybrid is really only for masochists
But if you want to get it right you have to train your A/V provider
In advance ...
Web Developer!
Thank You!
Questions?