I hope this will clear up some of the confusion around WebRTC media flows.
I guess this is one of the main reasons why I started with my new project of an Advanced WebRTC Architecture Course. In too many conversations I’ve had recently it seemed like people didn’t know exactly what happens with that WebRTC magic – what bits go where. While you can probably find that out by reading the specifications and the explanations around the WebRTC APIs or how ICE works, they all fail to consider the real use cases – the ones requiring media engines to be deployed.
So here we go.
In this article, I’ll be showing some of these flows. I made them part of the course – a whole lesson. If you are interested in learning more – then make sure to enroll to the course.
#1 – Basic P2P Call Direct WebRTC P2P callWe will start off with the basics and build on that as we move along.
Our entities will be colored in red. Signaling flows in green and media flows in blue.
What you see above is the classic explanation of WebRTC. Our entities:
What we have here is the classic VoIP (or WebRTC?) triangle. Signaling flows vertically towards the server but media flows directly across the browsers.
BTW – there’s some signaling going off from the browsers towards the STUN/TURN server for practically all types of scenarios. This is used to find the public IP address of the browsers at the very least. And almost always, we don’t draw this relationship (until you really need to fix a big, STUN seems obvious and too simple to even mention).
Summing this one up: nothing to write home about.
Moving on…
#2 – Basic Relay Call Basic WebRTC relay callThis is probably the main drawing you’ll see when ICE and TURN get explained.
In essence, the browsers couldn’t (or weren’t allowed) to reach each other directly with their media, so a third party needs to facilitate that for them and route the media. This is exactly why we use TURN servers in WebRTC (and other VoIP protocols).
This means that WebRTC isn’t necessarily P2P and P2P can’t be enforced – it is just a best effort thing.
So far so go. But somewhat boring and expected.
Let’s start looking at more interesting scenarios. Ones where we need a media server to handle the media:
#3 – WebRTC Media Server Direct Call, Centralized Signaling WebRTC Media Server Direct Call, Centralized SignalingNow things start to become interesting.
We’ve added a new entity into the mix – a media server. It can be used to record the calls, manage multiparty scenarios, gateway to other networks, do some other processing on the media – whatever you fancy.
To make things simple, we’ve dropped the relay via TURN. We will get to it in a moment, but for now – bear with me please.
MediaThe media now needs to flow through the media server. This may look like the previous drawing, where the media was routed through the TURN server – but it isn’t.
Where the TURN server relays the media without looking at it – and without being able to look at it (it is encrypted end-to-end); the Media Server acts as a termination point for the media and the WebRTC session itself. What we really see here is two separate WebRTC sessions – one from the browser on the left to the media server, and a second one from the media server to the browser on the right. This one is important to understand – since these are two separate WebRTC sessions – you need to think and treat them separately as well.
Another important note to make about media servers is that putting them on a public IP isn’t enough – you will still need a TURN server.
SignalingOn the signaling front, most assume that signaling continues as it always have. In which case, the media server needs to be controlled in some manner, presumably using a backend-to-backend signaling with the application server.
This is a great approach that keeps things simple with a single source of truth in the system, but it doesn’t always happen.
Why? Because we have APIs everywhere. Including in media servers. And these APIs are sometimes used (and even abused) by clients running browsers.
Which leads us to our next scenario:
#4 – WebRTC Media Server Direct Call, Split Signaling WebRTC Media Server Direct Call, Split SignalingThis scenario is what we usually get to when we add a media server into the mix.
Signaling will most often than not be done between the browser and the media server while at the same time we will have signaling between the browser and the application server.
This is easier to develop and start running, but comes with a few drawbacks:
Skip it if you can.
Now lets add back that STUN/TURN server into the mix.
#5 – WebRTC Media Server Call Relay WebRTC Media Server Call RelayThis scenario is actually #3 with one minor difference – the media gets relayed via TURN.
It will happen if the browsers are behind firewalls, or in special cases when this is something that we enforce for our own reasons.
Nothing special about this scenario besides the fact that it may well happen when your intent is to run scenario #3 – hard to tell your users which network to use to access your service.
#6 – WebRTC Media Server Call Partial Relay WebRTC Media Server Call Partial RelayJust like #5, this is also a derivative of #3 that we need to remember.
The relay may well happen only in one side of the media server – I hope you remember that each side is a WebRTC session on its own.
If you notice, I decided here to have signaling direct to the media server, but could have used the backend to backend signaling.
#7 – WebRTC Media Server and TURN Co-location WebRTC Media Server and TURN Co-locationThis scenario shows a different type of a decision making point. The challenge here is to answer the question of where to deploy the STUN/TURN server.
While we can put it as an independent entity that stands on its own, we can co-locate it with the media server itself.
What do we gain by this? Less moving parts. Scales with the media server. Less routing headaches. Flexibility to get media into your infrastructure as close to the user as possible.
What do we lose? Two different functions in one box – at a time when micro services are the latest tech fad. We can’t scale them separately and at times we do want to scale them separately.
Know Your FlowsThese are some of the decisions you’ll need to make if you go to deploy your own WebRTC infrastructure; and even if you don’t do that and just end up going for a communication API vendor – it is worthwhile understanding the underlying nature of the service. I’ve seen more than a single startup go work with a communication API vendor only to fail due to specific requirements and architectures that had to be put in place.
One last thing – this is 1 of 40 different lessons in my Advanced WebRTC Architecture Course. If you find this relevant to you – you should join me and enroll to the course. There’s an early bird discount valid until the end of this week.
The post How Media and Signaling flows look like in WebRTC? appeared first on BlogGeek.me.
Chad Hart joins the ClueCon Weekly Team to talk WebRTC
*It should be noted that the lip sync on this video is off due to bandwidth issues on our presenters side.
Where is the IMTC focusing it efforts when it comes to WebRTC?
[Bernard Aboba, who is IMTC Director and Principal Architect for Microsoft wanted to clarify a bit what the IMTC is doing in the WebRTC Activity Group. I was happy to give him this floor, clarifying a bit the tweet I shared in an earlier post]One of the IMTC’s core missions is to enhance interoperability in multimedia communications, with real-time video communications having been a focus of the organization since its inception. With IMTC’s membership including many companies within the video industry, IMTC has over the years dealt with a wide range of video interoperability issues, from simple 1:1 video scenarios to telepresence use cases involving multiple participants, each with multiple cameras and screens.
With WebRTC browsers now adding support for H.264/AVC as well as VP9, and support for advanced video functionality such as simulcast and scalable video coding (SVC) becoming available, the need for WebRTC video protocol and API interoperability testing has grown, particularly in scenarios implemented by video conferencing applications. As a result, the IMTC’s WebRTC Activity Group has been working to further interoperability testing between WebRTC browsers.
In the past, the IMTC has sponsored development of test suites, including a test suite for SIP over IPv6, and most recently a tool for testing interoperability of HEVC/H.265 scalable video coding. For SuperOp 2016, the WebRTC AG took on testing of WebRTC audio and video interoperability. So a logical next step was to work on development of automated WebRTC interoperability tests. Challenges include:
For its initial work, IMTC decided to focus on the first problem. To enable interoperability testing of the VP9 and H.264/AVC implementations now available in browsers, the IMTC supported Philipp Hancke (known to the community as “fippo”) in enhancing automated WebRTC interoperability tests, now available at https://github.com/fippo/testbed. Sample code used in the automated tests is available at https://github.com/webrtc/samples.
The interoperability tests depend on adapter.js, a Javascript “shim” library originally developed by the Chrome team to enable tests to be run on Chrome and Firefox. Support for VP9 and H.264/AVC has been rolled into adapter.js 2.0, as well as support for Edge (first added by fippo in October 2015). The testbed also depends on a merged fix (not yet released) in version 2.0.2. The latest adapter.js release as well as ongoing fixes is available at https://github.com/webrtc/adapter.
With the enhancements rolled into adapter.js 2.0, the shim library enables WebRTC developers to ship audio and video applications running across browsers using a single code base. At ClueCon 2016, Anthony Minessale of Freeswitch demonstrated the Verto client written to the WebRTC 1.0 API supporting audio and video interoperability between Chrome, Firefox and Edge.
–
Got questions or want to learn more about the IMTC and its involvement with WebRTC? Email the IMTC directly.
Planning on introducing WebRTC to your existing service? Schedule your free strategy session with me now.
The post IMTC: Supporting WebRTC Interoperability appeared first on BlogGeek.me.
Mod_kazoo had some API enhancements, mod_http_cache has GET and PUT from Azure Blob services, and mod_conference added a variable called conference_join_energy_level. The FreeSWITCH configuration audit is ongoing with initial minor commits and will continue throughout the year. If you are looking to volunteer to help with that or would like more information email [email protected] or join the Bug Hunt on Tuesdays at 12:00pm Central Time.
Join us Wednesdays at 12:00 CT for some more FreeSWITCH fun! And, head over to freeswitch.com to learn more about FreeSWITCH support.
New features that were added:
Improvements in build system, cross-platform support, and packaging:
The following bugs were squashed:
Yes you do. Sorry.
This is something I bumped into recently and was quite surprised it wasn’t obvious, which lead me to the conclusion that the WebRTC Architecture course I am launching is… mandatory. This was a company that had their media server on a public IP address, thinking that this should remove their need to run a TURN server. Apparently, the only thing it did was remove their connection rate.
It is high time I write about it here, as over the past year I actually saw 3 different ways in which vendors break their connectivity:
Newsflash: THIS ISN’T ENOUGH
I digress though. I want to explain why the first alternative is broken:
Why a public IP address for your media server isn’t enoughWith WebRTC, traffic goes peer to peer. Or at least it should:
But this doesn’t always work because one or both of the browsers are on private networks, so they don’t really have a public address to use – or don’t know it. If one of them has a public IP, then things should be simpler – the other end will direct traffic to that address, and from that “pinhole” that gets created, traffic can flow the other way.
The end result? If you put your media server on a public IP address – you’re set of success.
But the thing is you really aren’t.
There’s this notion of IT and security people that you should only open ports that need to be used. And since all traffic to the internet flows over HTTP(S); and HTTP(S) flows over TCP – you can just block UDP and be done with it.
Now, something that usually gets overlooked is that WebRTC uses UDP for its media traffic. Unless TURN relay over TCP/TLS is configured and necessary. Which sometimes it does. I asked a colleague of mine about the traffic they see, and got something similar to this distribution table:
With up to 20% of the sessions requiring TURN with TCP or TLS – it is no wonder a public IP configured on a media server just isn’t enough.
Oh, and while we’re talking security – I am not certain that in the long run, you really want your media server on the internet with nothing in front of it to handle nasty stuff like DDoS.
What should you do then?Whatever you do though, don’t rely on a public IP address in your media server to be enough.
The post Do you still need TURN if your media server has a public IP address? appeared first on BlogGeek.me.
Media servers, server-side media handling devices, continue to be a popular topic of discussion in WebRTC. One reason for this because they are the most complex elements in a VoIP architecture and that lends itself to differing approaches and misunderstandings. Putting WebRTC media servers in the cloud and reliably scaling them is even harder. Fortunately there are […]
The post WebRTC media servers in the Cloud: lessons learned (Luis López Fernández) appeared first on webrtcHacks.
Kurento or Jitsi; Kurento vs Jitsi – is the the ultimate head to head comparison for open source media servers in WebRTC?
Yes and no. And if you want an easy answer of “Kurento is the way to go” or “Jitsi will solve all of your headaches” then you’ve come to the wrong place. As with everything else here, the answer depends a lot on what it is you are trying to achieve.
Since this is something that get raised quite often these days by the people I chat with, I decided to share my views here. To do that, the best way I know is to start by explaining how I compartmentalized these two projects in my mind:
Jitsi VideobridgeThe Jitsi Videobridge is an SFU. It is an open source one, which is currently owned and maintained by Atlassian.
The acquisition of the Jitsi Videobridge serves Atlassian in two ways:
Here’s the intro of Jitsi from its github page:
Jitsi Videobridge is an XMPP server component that allows for multiuser video communication. Unlike the expensive dedicated hardware videobridges, Jitsi Videobridge does not mix the video channels into a composite video stream, but only relays the received video channels to all call participants. Therefore, while it does need to run on a server with good network bandwidth, CPU horsepower is not that critical for performance.
I emphasized the important parts for you. Here’s what they mean:
Put simply – Jitsi is an SFU with XMPP signaling.
If this is what you’re looking for then this baby is for you. If you don’t want/need an SFU or have other signaling protocol, better start elsewhere.
You can find outsourcing vendors who are happy to use Jitsi and have it customized or integrated to your use case.
KurentoKurento is a kind of an media server framework. This too is an open source one, but one that is maintained by Kurento Technologies.
With Kurento you can essentially build whatever you want when it comes to backend media processing: SFU, MCU, recording, transcoding, gateway, etc.
This is an advantage and a disadvantage.
An advantage because it means you can practically use it for any type of use case you have.
A disadvantage because there’s more work to be done with it than something that is single purpose and focused.
Kurento has its own set of vendors who are happy to support, customize and integrate it for you, one of which are the actual authors and maintainers of the Kurento code base.
Which one’s for you? Kurento or Jitsi?Both frameworks are very popular, with each having at the very least 10’s of independent installations and integrations done on top of them and running in production services.
Kurento or Jitsi? Kurento or Jitsi? Not always an easy choice, but here’s where I draw the line:
If what you need is a pure SFU with XMPP on top, then go with Jitsi. Or find some other “out of the box” SFU that you like.
If what you need is more complex, or necessitates more integration points, then you are probably better off using Kurento.
What about Janus?Janus is… somewhat tougher to explain.
Their website states that it is a “general purpose WebRTC Gateway”. So in my mind it will mostly fit into the role of a WebRTC-SIP gateway.
That said, I’ve seen more than a single vendor using it in totally other ways – anything from an SFU to an IOT gateway.
I need to see more evidence of use cases where production services end up using it for multiparty as opposed to a gateway component to suggest it as a solid alternative.
Oh – and there are other frameworks out there as well – open source or commercial.
Where can I learn more?Multiparty and server components are a small part of what is needed when going about building a WebRTC infrastructure for a communication service.
In the past few months, I’ve noticed a growing requests in challenges and misunderstandings of how and what WebRTC really is. People tend to focus on the obvious side of the browser APIs that WebRTC has, and forget to think about the backend infrastructure for it – something that is just as important, if not more.
It is why I’ve decided to launch an online WebRTC Architecture course that tackles these types of questions.
Course starts October 24, priced at $247 USD per student. If you enroll before October 10, there’s a $50 discount – so why wait?
The post Should you use Kurento or Jitsi for your multiparty WebRTC video conference product? appeared first on BlogGeek.me.
It was a quiet week in the code with some minor build updates and improvements. The FreeSWITCH configuration audit has begun with initial minor commits and will continue throughout the year. If you are looking to volunteer to help with that or would like more information email [email protected] or join the Bug Hunt on Tuesdays at 12:00pm Central Time.
Join us Wednesdays at 12:00 CT for some more FreeSWITCH fun! And, head over to freeswitch.com to learn more about FreeSWITCH support.
Improvements in build system, cross platform support, and packaging:
The following bugs were squashed:
No. Yes. Don’t know.
I’ve recently read an article at iSchool@Syracuse. For lack of a better term on my part, pundits opining about the decentralized web.
It is an interesting read. Going through the opinions there, you can divide the crowd into 3 factions:
I am… somewhat split across all of these three groups.
#1 – Privacy, Gatekeepers and MonopoliesLike any other person, I want privacy. On the other hand, I want security, which in many cases (and especially today) comes at the price of privacy. I also want convenience, and at the age of artificial intelligence and chat bots – this can easily mean less privacy.
As for governments and monopolies – I don’t think these will change due to a new protocol or a decentralized web. The web started as something decentralized and utopian to some extent. It degraded to what it is today because governments caught on and because companies grew inside the internet to become monopolies. Can we redesign it all in a way that will not allow for governments to rule over the data going into them or for monopolies to not exist? I doubt it.
I am taking part now in a few projects where location matters. Where you position your servers, how you architect your network, and even how you communicate your intent with governments – all these can make or break your service. I just can’t envision how protocols can change that in a global scale – and how the forces that be that need to promote and push these things will actively do so.
I think it is a good thing to strive for, but something that is going very challenging to achieve:
Yes. Our web is client-server for the most part, with browsers getting their data fix from backend servers.
We now have technologies that can work differently (WebRTC’s data channel is one of them, and there are others still).
We can and should work on making our infrastrucuture more robust. More impregnable to malicious attackers and prone to errors. We should make it scale better. And yes. Decentralization is usually a good design pattern to achieve these goals.
But if at the end of the day, the decentralized web is only about maintaining the same user experience, then this is just a slow evolution of what we’re already doing.
Tech is great. I love tech. Most people don’t really care.
#3 – We just don’t knowAs with many other definitions out there, there’s no clear definition of what the decentralized web is or should be. Just a set of opinions by different pundits – most with an agenda for putting out that specific definition.
I really don’t know what that is or what it should be. I just know that our web today is centralized in many ways, but in other ways it is already rather decentralized. The idea that I have this website hosted somewhere (I am clueless as to where), while I write these words from my home in Israel, it is being served either directly or from a CDN to different locations around the globe – all done through a set of intermediaries – some of which I specifically selected (and pay for or use for free) – to me that’s rather decentralized.
–
At the end of the day, the work being done by researchers for finding ways to utilize our existing protocols to offer decentralized, robust services or to define and develop new protocols that are inherently decentralized is fascinating. I’ve had my share of it in my university days. This field is a great place to research and learn about networks and communications. I can’t wait to see how these will evolve our every day networks.
The post Will there ever be a decentralized web? appeared first on BlogGeek.me.
The FreeSWITCH 1.6.10 release is here!
This is also a routine maintenance release. Change Log and source tarball information below.
Release files are located here:
New features that were added:
Improvements in build system, cross platform support, and packaging:
The following bugs were squashed:
The features this week include added detection time to beep event in mod_avmd and event_channel_broadcaster API in mod_commands.
Join us Wednesdays at 12:00 CT for some more FreeSWITCH fun! And, head over to freeswitch.com to learn more about FreeSWITCH support.
New features that were added:
Improvements in build system, cross platform support, and packaging:
The following bugs were squashed:
This week we had some great improvements to mod_verto, mod_sofia, mod_conference, mod_avmd, and mod_v8. These changes include support and minor configuration improvements.
Join us Wednesdays at 12:00 CT for some more FreeSWITCH fun! And, head over to freeswitch.com to learn more about FreeSWITCH support.
New features that were added:
Improvements in build system, cross platform support, and packaging:
The following bugs were squashed:
Phosfluorescently utilize future-proof scenarios whereas timely leadership skills. Seamlessly administrate maintainable quality vectors whereas proactive mindshare.
Dramatically plagiarize visionary internal or "organic" sources via process-centric. Compellingly exploit worldwide communities for high standards in growth strategies.
Wow, this most certainly is a great a theme.
Donec sed odio dui. Nulla vitae elit libero, a pharetra augue. Nullam id dolor id nibh ultricies vehicula ut id elit. Integer posuere erat a ante venenatis dapibus posuere velit aliquet.
Donec sed odio dui. Nulla vitae elit libero, a pharetra augue. Nullam id dolor id nibh ultricies vehicula ut id elit. Integer posuere erat a ante venenatis dapibus posuere velit aliquet.