Posted: Mon Oct 23, 2006 9:43 pm Post subject: [Asterisk-video] Re: [Iaxclient-devel] Video codec negotiati
On 10/23/06, Mihai Balea <mihai@hates.ms> wrote:
Quote:
Hello all,
As you may already know, I am working on adding video support to
iaxclient.
It appears however that IAX's support for video codec negotiation is
lacking. I am no IAX expert, but from what I can see, there are to
ways of doing codec negotiation in IAX:
- use the FORMAT and CAPABILITY IEs. This allows you to specify one
preferred codec and a list of acceptable codecs. This is the way
iaxclient does codec negotiation at this time.
- use the CODEC PREFS IE. This allows you to specify a list of
acceptable codecs, in order of preference.
The problem with video is that you need to negotiate two codecs at
the same time (one for video and one for audio). Unfortunately, none
of the above mentioned methods are suitable for this. The way I see
it, there are several ways of addressing this:
1. use a VIDEO FORMAT IE, in addition to FORMAT and CAPABILITY. The
CAPABILITY IE will contain acceptable video codecs as well as audio.
2. use a VIDEO CODEC PREFS IE in addition to the current CODEC PREFS
IE. This will more flexible since it allows for video codec
preference order.
3. use the current FORMAT and CAPABILITY and change the semantics of
FORMAT to allow a video codec as well as an audio codec. This might
be easiest to implement.
As a long term solution, I'm inclined to go with #2 since it provides
the maximum amount of flexibility. However I would like to hear any
suggestions you might have regarding this issue...
Thanks in advance,
Mihai
I'm pasting a message I've written a while ago on the iax client list
- i think it's very much in tune to what's being discussed.
Background on this discussion is trying to get an video echo call
iaxclient <-> Asterisk to work.
- when testing a SIP echo video call (which works audio and video) I
see that the SIP client starts off with audio only
- at some point in the future the client adds video frames to the mix
(user says - yes, start video)
This does not look like it could be supported by the current version
of iaxclient.
iaxclient tries to do the negociation at call set-up time (i.e. when
IAX NEW and ACCEPT frames are sent and received) and it doesn't change
it in mid call.
I am wondering if Asterisk is not doing a codec "renegociation" in mid
call and we're trying to mix apples and oranges by trying to persuade
chan_iax into doing something it can't...
I'm referring to this fragment from chan_iax2.c (in function
socket_read, asterisk version 1.2.something):
if (f.frametype == AST_FRAME_VIDEO) {
if (f.subclass != iaxs[fr->callno]->videoformat) {
ast_log(LOG_DEBUG, "Ooh, video format
changed to %d\n", f.subclass & ~0x1);
iaxs[fr->callno]->videoformat =
f.subclass & ~0x1;
}
}
I don't see any logic in this code as long as chan_iax would not
support mid-call codec changes.
from my point of view, the ideal iaxclient would
- advertise accepting video frames even if a camera is not present
- advertise accepting audio frames even if microphone is not present
(think monitor, i.e. read voice mail where you only input DTMF digits)
- if a call from the call list has a ulaw codec set and a (i imagine
big frame) video frame arrives from the iaxclient remote peer, i think
the local peer should try and match the codec for the received video
frame and if it succeeds mask the codec capabilities with this codec
(this would allow iax peers to turn cameras off and on whilst in mid
call).
Posted: Tue Oct 24, 2006 8:19 am Post subject: [Asterisk-video] Re: [Iaxclient-devel] Video codec negotiati
On Oct 24, 2006, at 2:43 AM, Cristian Draghici wrote:
Quote:
- when testing a SIP echo video call (which works audio and video) I
see that the SIP client starts off with audio only
- at some point in the future the client adds video frames to the mix
(user says - yes, start video)
This does not look like it could be supported by the current version
of iaxclient.
iaxclient tries to do the negociation at call set-up time (i.e. when
IAX NEW and ACCEPT frames are sent and received) and it doesn't change
it in mid call.
I am wondering if Asterisk is not doing a codec "renegociation" in mid
call and we're trying to mix apples and oranges by trying to persuade
chan_iax into doing something it can't...
Iaxclient definitely does not support codec re-negotiation right
now. As a matter of fact I am not sure if the IAX2 protocol supports
it (somebody please correct me if I'm wrong). We could probably
delay sending/accepting video but the negotiation has to happen at
call setup.
As an aside, I belive in SIP this is done through reINVITEs. Not
many SIP endpoints support codec changes in mid-call correctly.
Anyway, I don't see a similar mechanism in IAX2
Quote:
from my point of view, the ideal iaxclient would
- advertise accepting video frames even if a camera is not present
- advertise accepting audio frames even if microphone is not present
(think monitor, i.e. read voice mail where you only input DTMF digits)
This makes sense. Not sure what is the behavior right now, but if we
don't do it, then it's a bug that need to be fixed
Quote:
- if a call from the call list has a ulaw codec set and a (i imagine
big frame) video frame arrives from the iaxclient remote peer, i think
the local peer should try and match the codec for the received video
frame and if it succeeds mask the codec capabilities with this codec
(this would allow iax peers to turn cameras off and on whilst in mid
call).
Not sure I get you here, but I believe you're saying that if a video
frame arrives in the middle of an audio conversation, we should be
able to process it (provided we have the correct codec, of course).
We can look into it, this might be something that we could support.
Another related feature would be sending a "holding" frame when no
video is present (or when the user is in "mute" mode). There are two
ways of doing this:
- easy: just keep resending the last frame captured or another raw
image.
- complicated: use IAX2 Image frame and have iaxclient display that
static image.
Posted: Wed Oct 25, 2006 5:54 am Post subject: [Asterisk-video] Re: [Iaxclient-devel] Video codec negotiati
On Oct 25, 2006, at 12:44 AM, Cristian Draghici wrote:
Quote:
I've written a very basic IAX decoder a while back and I remember that
the RFC IAX found on the net are somewhat old; you best bet being
reading the Asterisk sources.
The latest specs that I know of are these:
>
> As an aside, I belive in SIP this is done through reINVITEs. Not
> many SIP endpoints support codec changes in mid-call correctly.
> Anyway, I don't see a similar mechanism in IAX2
I think it would make sense but if I'm the only one thinking it, tough
luck, eh :-)
I guess it's more the case of people not caring enough to implement
it... mostly due to lack of resources.
If you need something like this for your projects, you should go
ahead and implement it yourself... I bet most parties involved would
be glad to incorporate it... I know I would :)
Quote:
But this is somewhat related to what i call codec negociation.
Imagine this:
- peer A says I can handle ulaw
- peer B says I can handle ulaw, h264
- peer A calls peer B audio call
- user at peer A starts camera
- peer A notices peer B accepts h264 frames and is able to encode them
so starts sending them over
At this point a 2 way ulaw call turned into a 2 way ulaw + 1 way
h264 call.
If A knows h264, it should advertise it in the beginning. This way B
can select it and start a 2-way audio, 1-way video call immediately.
When A turns on his camera, the call becomes 2-way video.
A could also send a holding frame while its camera is off.
Quote:
>
> Another related feature would be sending a "holding" frame when no
> video is present (or when the user is in "mute" mode). There are two
> ways of doing this:
> - easy: just keep resending the last frame captured or another raw
> image.
> - complicated: use IAX2 Image frame and have iaxclient display that
> static image.
I think easy is better as I don't see any functional difference in
between the two.
The second option would just save some bandwidth, other than that
they would be identical from a user's point of view.
You can post new topics in this forum You can reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum