[asterisk-dev] How about to support multi video tracks of the webrtc sfu?

[Xiemin Chen]

Anyone can help? Thanks very much.

Hi there,
I was told that it's not support by asterisk to have multi video tracks of
one client in the following post
<https://community.asterisk.org/t/asterisk-webrtc-sfu-does-it-support-multi-video-track/76757>.
How about if I want to achieve this functionality? Is there any technical
limitation?

[Matt Fredrickson]

Post by Xiemin Chen
Anyone can help? Thanks very much.

Hi there,
I was told that it's not support by asterisk to have multi video tracks of one client in the following post. How about if I want to achieve this functionality? Is there any technical limitation?

I think Josh's response on the community forums implied that there
probably *are* going to be technical limitations. I don't think
anybody's tried to prototype a single client with multiple resolutions
of the same stream from the same endpoint at this point.

So I guess maybe trying to build a browser app prototype that can
produce that scenario first would be a good start. Then you'll need
to see how that's represented in SDP, to figure out how that lines up
with Asterisk's multi-stream capabilities and make sure that it tries
to do the right thing from its side.

As Josh mentioned, there's gonna be a bit of work to make things happy
inside of Asterisk. It sounds like a really fun project though, and
if you manage to get somewhere on it I'd love to hear about your
progress.

Good luck and best wishes!
--
Matthew Fredrickson
Digium - A Sangoma Company | Asterisk Project Lead
445 Jan Davis Drive NW - Huntsville, AL 35806 - USA

[Xiemin Chen]

I just made it works with multi video tracks of the echo stream demo by a
few lines of source code change in apps/app_stream_echo.c:

249c249
< // continue;
---

continue;

278c278
< } while (0);
---

} while (--n);

Mean while I have a little change in native client code to have two video
tracks in one stream at the same time.

So I think it's not a lot of code change if needs to do this task. I want
to know your ideas, thanks very much.

Sincerely
Xiemin

Post by Xiemin Chen
Anyone can help? Thanks very much.

Hi there,
I was told that it's not support by asterisk to have multi video tracks

of one client in the following post. How about if I want to achieve this
functionality? Is there any technical limitation?
I think Josh's response on the community forums implied that there
probably *are* going to be technical limitations. I don't think
anybody's tried to prototype a single client with multiple resolutions
of the same stream from the same endpoint at this point.
So I guess maybe trying to build a browser app prototype that can
produce that scenario first would be a good start. Then you'll need
to see how that's represented in SDP, to figure out how that lines up
with Asterisk's multi-stream capabilities and make sure that it tries
to do the right thing from its side.
As Josh mentioned, there's gonna be a bit of work to make things happy
inside of Asterisk. It sounds like a really fun project though, and
if you manage to get somewhere on it I'd love to hear about your
progress.
Good luck and best wishes!
--
Matthew Fredrickson
Digium - A Sangoma Company | Asterisk Project Lead
445 Jan Davis Drive NW - Huntsville, AL 35806 - USA
--
_____________________________________________________________________
-- Bandwidth and Colocation Provided by http://www.api-digital.com --
https://www.asterisk.org/community/astricon-user-conference
asterisk-dev mailing list
http://lists.digium.com/mailman/listinfo/asterisk-dev

[Joshua C. Colp]

Post by Xiemin Chen
I just made it works with multi video tracks of the echo stream demo by a
249c249
< // continue;
---

Post by Xiemin Chen
continue;

278c278
< } while (0);
---

Post by Xiemin Chen
} while (--n);

Mean while I have a little change in native client code to have two video
tracks in one stream at the same time.
So I think it's not a lot of code change if needs to do this task. I want
to know your ideas, thanks very much.

The stream echo application is much simpler than the SFU, as it only has to do basic renegotiation/mapping. The code that the SFU relies upon is more complex as it has to keep track of the different streams and their mappings across all of the channels. This was done with a single source in mind, and the mapping process (and stream names) reflect that. I don't think the code will work out of the box, and until someone dives in there's no telling what needs to be really changed.
--
Joshua C. Colp
Digium - A Sangoma Company | Senior Software Developer
445 Jan Davis Drive NW - Huntsville, AL 35806 - US
Check us out at: www.digium.com & www.asterisk.org

[Xiemin Chen]

---------- Forwarded message ---------
From: Xiemin Chen <***@gmail.com>
Date: 2018幎11月25日呚日 䞋午11:19
Subject: Re: [asterisk-dev] How about to support multi video tracks of the
webrtc sfu?
To: <asterisk-***@lists.digium.com>


It just workd with the two user's video-conference which both of them have
two video track separately. Now each one can see other's video and screen
sharing.

For the test purpose, I just hard code the to_channel mapping in
the bridge_channel_handle_write() then everything seems go fine.

Indeed, the code cannot run out-of-the-box and I want to know your ideas
about, anything difficult to have the correct mapping of the to_channel
vector? Or do I have any extra points missed to know its difficulty?

if (fr->stream_num == 6)
num = 3;
else if (fr->stream_num == 5)
num = 4;
else if (fr->stream_num == 3)
num = 3;
else if (fr->stream_num == 2)
num = 4;


Sincerely
Xiemin

Post by Joshua C. Colp

Post by Xiemin Chen
I just made it works with multi video tracks of the echo stream demo by a
249c249
< // continue;
---

Post by Xiemin Chen
continue;

278c278
< } while (0);
---

Post by Xiemin Chen
} while (--n);

Mean while I have a little change in native client code to have two video
tracks in one stream at the same time.
So I think it's not a lot of code change if needs to do this task. I want
to know your ideas, thanks very much.

The stream echo application is much simpler than the SFU, as it only has
to do basic renegotiation/mapping. The code that the SFU relies upon is
more complex as it has to keep track of the different streams and their
mappings across all of the channels. This was done with a single source in
mind, and the mapping process (and stream names) reflect that. I don't
think the code will work out of the box, and until someone dives in there's
no telling what needs to be really changed.
--
Joshua C. Colp
Digium - A Sangoma Company | Senior Software Developer
445 Jan Davis Drive NW - Huntsville, AL 35806 - US
Check us out at: www.digium.com & www.asterisk.org
--
_____________________________________________________________________
-- Bandwidth and Colocation Provided by http://www.api-digital.com --
https://www.asterisk.org/community/astricon-user-conference
asterisk-dev mailing list
http://lists.digium.com/mailman/listinfo/asterisk-dev

[Joshua C. Colp]

Post by Xiemin Chen
---------- Forwarded message ---------
Date: 2018年11月25日周日 下午11:19
Subject: Re: [asterisk-dev] How about to support multi video tracks of the
webrtc sfu?
It just workd with the two user's video-conference which both of them have
two video track separately. Now each one can see other's video and screen
sharing.
For the test purpose, I just hard code the to_channel mapping in
the bridge_channel_handle_write() then everything seems go fine.
Indeed, the code cannot run out-of-the-box and I want to know your ideas
about, anything difficult to have the correct mapping of the to_channel
vector? Or do I have any extra points missed to know its difficulty?
if (fr->stream_num == 6)
num = 3;
else if (fr->stream_num == 5)
num = 4;
else if (fr->stream_num == 3)
num = 3;
else if (fr->stream_num == 2)
num = 4;

It's not something I've thought about so I don't really know what things may occur - I can say, though, that the stream names to actually calculate the proper mapping may collide and cause a problem.
--
Joshua C. Colp
Digium - A Sangoma Company | Senior Software Developer
445 Jan Davis Drive NW - Huntsville, AL 35806 - US
Check us out at: www.digium.com & www.asterisk.org