On Mon, 2017-12-11 at 10:13 +0000, David Woodhouse wrote: > On Sun, 2017-12-10 at 21:02 +0000, David Woodhouse wrote: > > I'm thinking of following the precedent set by FsShmTransmitter and > > abusing the host and username fields of the FsCandidate, except not > > with filenames as FsShmTransmitter does it, but with pipeline > > descriptions to be handled by gst_parse_launch(). > > OK, so that much works, but I can't used named elements; those have to > have been created in the *same* pipeline description. So this code > doesn't work because there's no way to make it find "MyTestSrc" AFAICT:
So I got this working. I can create the named element in the FsAppTransmitter pipeline, then I can find it again from the outside with gst_bin_get_by_name(). My own protocol code can now feed Opus frames up from the GstAppSrc, and receive them from the GstAppSink, and I can actually have a conversation. Yay! However... I need to do a bunch of stuff for myself. There are timestamp fields in the wire protocol, which I need to manually keep in sync with the timestamps on the GstBuffers, and I also need a jitter buffer. Now that now my life isn't *entirely* dominated by CPU vulnerability work I'm starting to take a look again. It occurred to me that I'd actually be better off faking RTP, and just copying my protocol's timestamps into the corresponding field of an RTP packet. That way I can avoid reinventing the wheel. Before I do the step of faking RTP myself, I started with purely adding rtpopuspay/rtpopusdepay in my pipelines and switching to fsrtpconference. And now I'm back to "I am clueless and I don't know why this doesn't work mode" and asking for help again... :) I'm still working on getting permission to actually publish this PRPL under LGPL, but below is a pseudo-patch which shows what was working, and what I've done. I can probably do this in the dummy PRPL as I did before, if anyone really wants to be able to test it locally. The GStreamer pipeline is at http://david.woodhou.se/conf.svg When I press a key it does correctly send DTMF. But although the audio from the microphone *looks* like it's hooked up, nothing is arriving. And the incoming audio doesn't seem to be hooked up at all. And I never see 'farstream-recv-codec-changed'. Any further clues would be, as before, most gratefully received. static void on_audio_state(ChimeCall *call, ChimeAudioState audio_state, struct chime_chat *chat) { purple_debug(PURPLE_DEBUG_INFO, "chime", "Audio state %d\n", audio_state); const gchar *name = chime_call_get_alert_body(chat->call); if (audio_state == CHIME_AUDIO_STATE_AUDIO_MUTED && chat->media) { purple_media_stream_info(chat->media, PURPLE_MEDIA_INFO_MUTE, "chime", name, FALSE); } else if (audio_state == CHIME_AUDIO_STATE_AUDIO && chat->media) { purple_media_stream_info(chat->media, PURPLE_MEDIA_INFO_UNMUTE, "chime", name, FALSE); } else if (audio_state == CHIME_AUDIO_STATE_AUDIO && !chat->media) { PurpleMediaManager *mgr = purple_media_manager_get(); chat->media = purple_media_manager_create_media(purple_media_manager_get(), chat->conv->account, - "fsrawconference", + "fsrtpconference", name, TRUE); if (chat->media) { - const gchar *caps = "audio/x-raw,format=(string)S16LE,layout=(string)interleaved,channels=(int)1"; + const gchar *caps = "application/x-rtp,media=(string)audio,payload=(int)96,encoding-name=(string)OPUS"; gboolean r = purple_media_add_stream(chat->media, "chime", name, PURPLE_MEDIA_AUDIO, TRUE, "app", 0, NULL); gchar *srcname = g_strdup_printf("chime_src_%p", call); gchar *sinkname = g_strdup_printf("chime_sink_%p", call); /* Without the capsfilter (even though there's another capsfilter almost immediately after it in the pipeline, it doesn't work */ - gchar *srcpipe = g_strdup_printf("appsrc name=%s format=time caps=audio/x-opus,channel-mapping-family=0 ! opusdec ! capsfilter caps=%s", - srcname, caps); - gchar *sinkpipe = g_strdup_printf("opusenc bitrate=16000 bitrate-type=vbr ! appsink name=%s async=false", sinkname); + gchar *srcpipe = g_strdup_printf("appsrc name=%s format=time caps=audio/x-opus,channel-mapping-family=0 ! rtpopuspay ! capsfilter caps=%s", srcname, caps); + gchar *sinkpipe = g_strdup_printf("rtpopusdepay ! appsink name=%s async=false", sinkname); PurpleMediaCandidate *cand = purple_media_candidate_new(NULL, 1, PURPLE_MEDIA_CANDIDATE_TYPE_HOST, PURPLE_MEDIA_NETWORK_PROTOCOL_UDP, sinkpipe, 0); g_object_set(cand, "username", srcpipe, NULL); g_free(sinkpipe); g_free(srcpipe); GList *cands = g_list_append (NULL, cand); GList *codecs = g_list_append(NULL, - purple_media_codec_new(1, caps, PURPLE_MEDIA_AUDIO, 0)); + purple_media_codec_new(96, "OPUS", PURPLE_MEDIA_AUDIO, 0)); +// g_object_set(codecs->data, "channels", 1, NULL); + purple_media_codec_add_optional_parameter(codecs->data, "farstream-recv-profile", "rtpopusdepay ! opusdec"); + purple_media_codec_add_optional_parameter(codecs->data, "farstream-send-profile", "opusenc ! rtpopuspay"); +// purple_media_codec_add_optional_parameter(codecs->data, "sprop-stereo", "0"); +// purple_media_codec_add_optional_parameter(codecs->data, "stereo", "0"); + purple_media_get_local_candidates(chat->media, "chime", name); purple_media_add_remote_candidates(chat->media, "chime", name, cands); + purple_media_set_send_codec(chat->media, "chime", codecs->data); purple_media_set_remote_codecs(chat->media, "chime", name, codecs); GstElement *pipeline = purple_media_manager_get_pipeline(mgr); GstElement *appsrc = gst_bin_get_by_name(GST_BIN(pipeline), srcname); GstElement *appsink = gst_bin_get_by_name(GST_BIN(pipeline), sinkname); g_free(srcname); g_free(sinkname); gst_app_src_set_size(GST_APP_SRC(appsrc), -1); gst_app_src_set_max_bytes(GST_APP_SRC(appsrc), 100); gst_app_src_set_stream_type(GST_APP_SRC(appsrc), GST_APP_STREAM_TYPE_STREAM); // gst_base_src_set_live(GST_BASE_SRC(appsrc), TRUE); chime_call_install_gst_app_callbacks(chat->call, GST_APP_SRC(appsrc), GST_APP_SINK(appsink)); g_signal_connect(chat->media, "state-changed", G_CALLBACK(call_media_changed), chat); g_signal_connect(chat->media, "stream-info", G_CALLBACK(call_stream_info), chat); purple_media_stream_info(chat->media, PURPLE_MEDIA_INFO_ACCEPT, "chime", name, FALSE); GST_DEBUG_BIN_TO_DOT_FILE(GST_BIN(purple_media_manager_get_pipeline(mgr)), GST_DEBUG_GRAPH_SHOW_ALL, "chime conference graph"); } } }
smime.p7s
Description: S/MIME cryptographic signature
_______________________________________________ Farstream-devel mailing list Farstream-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/farstream-devel