技术专栏
nvidia xavier nx平台GStreamer pipeline挂起问题调试1
1. 前言
在nvidia xavier nx平台使用GStreamer插件时，
出现一些异常情况，这里记录下调试过程
2.挂起问题
我设计了一个pipeline，使用两个不同的摄像头在我的基于Python/Qt的应用程序中组成输出。
一切正常，但应用程序在几个小时或几分钟后就会挂起，
使用调试级别3运行gstreamer会暴露以下警告
0:00:01.016107283 14314 0x559bdfe590 WARN v4l2bufferpool gstv4l2bufferpool.c:790:gst_v4l2_buffer_pool_start:<source_2:pool:src> Uncertain or not enough buffers, enabling copy threshold
0:00:01.923523511 14314 0x559bdfe590 WARN v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<source_2:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:00:01.939974401 14314 0x559bdfe590 WARN v4l2src gstv4l2src.c:976:gst_v4l2src_create:<source_2> lost frames detected: count = 230 - ts: 0:00:00.919337027
0:00:02.990022327 14314 0x559bdfe590 WARN v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<source_2:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:00:03.006565985 14314 0x559bdfe590 WARN v4l2src gstv4l2src.c:976:gst_v4l2src_create:<source_2> lost frames detected: count = 136 - ts: 0:00:01.985947187
0:00:04.289911178 14314 0x559bdfe590 WARN v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<source_2:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:00:04.306536499 14314 0x559bdfe590 WARN v4l2src gstv4l2src.c:976:gst_v4l2src_create:<source_2> lost frames detected: count = 205 - ts: 0:00:03.285886027
0:00:14.463057820 14314 0x559bdfe590 WARN queue gstqueue.c:988:gst_queue_handle_sink_event:<queue2> error: Internal data stream error.
0:00:14.463200958 14314 0x559bdfe590 WARN queue gstqueue.c:988:gst_queue_handle_sink_event:<queue2> error: streaming stopped, reason not-linked (-1)
0:00:14.463468224 14314 0x559bdfe540 WARN queue gstqueue.c:988:gst_queue_handle_sink_event:<queue0> error: Internal data stream error.
0:00:14.463529633 14314 0x559bdfe540 WARN queue gstqueue.c:988:gst_queue_handle_sink_event:<queue0> error: streaming stopped, reason not-linked (-1)
0:00:14.607746627 14314 0x559bdfe540 WARN v4l2bufferpool gstv4l2bufferpool.c:790:gst_v4l2_buffer_pool_start:<source_1:pool:src> Uncertain or not enough buffers, enabling copy threshold
0:00:15.610458006 14314 0x559bdfe540 WARN queue gstqueue.c:988:gst_queue_handle_sink_event:<queue4> error: Internal data stream error.
0:00:15.610592792 14314 0x559bdfe540 WARN queue gstqueue.c:988:gst_queue_handle_sink_event:<queue4> error: streaming stopped, reason not-linked (-1)
0:00:15.610765401 14314 0x559bdfe540 WARN queue gstqueue.c:988:gst_queue_handle_sink_event:<queue5> error: Internal data stream error.
0:00:15.610828698 14314 0x559bdfe540 WARN queue gstqueue.c:988:gst_queue_handle_sink_event:<queue5> error: streaming stopped, reason not-linked (-1)
0:00:15.703272958 14314 0x559bdfe540 WARN v4l2bufferpool gstv4l2bufferpool.c:790:gst_v4l2_buffer_pool_start:<source_1:pool:src> Uncertain or not enough buffers, enabling copy threshold
0:00:17.365679776 14314 0x559bdfe540 WARN queue gstqueue.c:988:gst_queue_handle_sink_event:<queue8> error: Internal data stream error.
0:00:17.366101444 14314 0x559bdfe540 WARN queue gstqueue.c:988:gst_queue_handle_sink_event:<queue8> error: streaming stopped, reason not-linked (-1)
0:00:17.366437031 14314 0x559bdfe540 WARN queue gstqueue.c:988:gst_queue_handle_sink_event:<queue9> error: Internal data stream error.
0:00:17.366741162 14314 0x559bdfe540 WARN queue gstqueue.c:988:gst_queue_handle_sink_event:<queue9> error: streaming stopped, reason not-linked (-1)
0:00:17.466034826 14314 0x559bdfe590 WARN v4l2bufferpool gstv4l2bufferpool.c:790:gst_v4l2_buffer_pool_start:<source_2:pool:src> Uncertain or not enough buffers, enabling copy threshold
0:01:07.267125833 14314 0x559bdfe540 WARN v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<source_1:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:01:07.320018089 14314 0x559bdfe540 WARN v4l2src gstv4l2src.c:976:gst_v4l2src_create:<source_1> lost frames detected: count = 11 - ts: 0:00:49.825705798
0:01:18.469308144 14314 0x559bdfe590 WARN v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<source_2:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:01:18.486142300 14314 0x559bdfe590 WARN v4l2src gstv4l2src.c:976:gst_v4l2src_create:<source_2> lost frames detected: count = 255 - ts: 0:01:01.014083542
0:02:35.649454180 14314 0x559bdfe590 WARN v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<source_2:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:02:35.666158884 14314 0x559bdfe590 WARN v4l2src gstv4l2src.c:976:gst_v4l2src_create:<source_2> lost frames detected: count = 168 - ts: 0:02:18.194158446
0:02:40.715948578 14314 0x559bdfe590 WARN v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<source_2:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:02:40.732665022 14314 0x559bdfe590 WARN v4l2src gstv4l2src.c:976:gst_v4l2src_create:<source_2> lost frames detected: count = 71 - ts: 0:02:23.260606902
0:03:03.117321379 14314 0x559bdfe540 WARN v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<source_1:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:03:03.157331221 14314 0x559bdfe540 WARN v4l2src gstv4l2src.c:976:gst_v4l2src_create:<source_1> lost frames detected: count = 1 - ts: 0:02:45.663025742
0:03:32.122298850 14314 0x559bdfe540 WARN v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<source_1:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:03:32.122536421 14314 0x559bdfe540 WARN v4l2allocator gstv4l2allocator.c:1349:gst_v4l2_allocator_dqbuf:<source_1:pool:src:allocator> V4L2 provided buffer has bytesused 0 which is too small to include data_offset 0gstreamer pipeline如下：
gst-launch-1.0 -e \
v4l2src io-mode=2 device=/dev/video0 name=source_1 \
v4l2src io-mode=2 device=/dev/video0 name=source_2 \
nvcompositor name=comp sink_0::width=1920 sink_0::height=1080 sink_1::xpos=1280 sink_1::width=640 sink_1::height=360 \
source_1. ! video/x-raw,format=YUY2,width=1920,height=1080,framerate=30/1 ! nvvidconv ! 'video/x-raw(memory:NVMM)' ! tee name=tee_source_1 \
tee_source_1. ! queue ! comp. \
tee_source_1. ! queue max-size-buffers=2 leaky=downstream ! nvvidconv ! video/x-raw,format=BGRx ! appsink max-buffers=10 drop=true emit-signals=true sync=false name=sink_primary \
source_2. ! video/x-raw,format=YUY2,width=1920,height=1080,framerate=30/1 ! nvvidconv ! 'video/x-raw(memory:NVMM),width=640,height=360' ! tee name=tee_source_2 \
tee_source_2. ! queue ! comp. \
tee_source_2. ! queue max-size-buffers=2 leaky=downstream ! nvvidconv ! video/x-raw,format=BGRx,width=640,height=360 ! appsink max-buffers=10 drop=true emit-signals=true sync=false name=sink_secondary \
comp. ! 'video/x-raw(memory:NVMM)' ! nvvidconv ! omxh265enc ! tee name=recording_tee \
recording_tee. ! queue ! rtph265pay config-interval=10 pt=96 ! udpsink host=127.0.0.1 port=5000 \
recording_tee. ! queue ! splitmuxsink location=cam-recording-%03d.mp4 max-size-time=9000000000000 max-files=96我认为问题是，应用接收器需要很长时间来处理，最终在管道中阻塞了一些东西，但我不确定
3. 尝试使用nvv4l2h265enc
建议尝试nvv4l2camerasrc。
您可以重新构建它以支持YUY2格式。
参考此补丁
diff --git a/gst-nvv4l2camera/gstnvv4l2camerasrc.cpp b/gst-nvv4l2camera/gstnvv4l2camerasrc.cpp
index 26ad70c..ad0a512 100644
--- a/gst-nvv4l2camera/gstnvv4l2camerasrc.cpp
+++ b/gst-nvv4l2camera/gstnvv4l2camerasrc.cpp
@@ -61,7 +61,7 @@
"video/x-raw(memory:NVMM), " \
"width = (int) [ 1, MAX ], " \
"height = (int) [ 1, MAX ], " \
- "format = (string) { UYVY }, " \
+ "format = (string) { YUY2 }, " \
"interlace-mode = (string) { progressive, interlaced }, " \
"framerate = (fraction) [ 0, MAX ];"
@@ -194,7 +194,7 @@ gst_nv_memory_allocator_alloc (GstAllocator * allocator,
input_params.height = self->height;
}
input_params.layout = NvBufferLayout_Pitch;
- input_params.colorFormat = NvBufferColorFormat_UYVY;
+ input_params.colorFormat = NvBufferColorFormat_YUYV;
input_params.payloadType = NvBufferPayload_SurfArray;
input_params.nvbuf_tag = NvBufferTag_CAMERA;
@@ -229,7 +229,7 @@ gst_nv_memory_allocator_alloc (GstAllocator * allocator,
params.layout = NVBUF_LAYOUT_PITCH;
params.memType = NVBUF_MEM_DEFAULT;
params.gpuId = 0;
- params.colorFormat = NVBUF_COLOR_FORMAT_UYVY;
+ params.colorFormat = NVBUF_COLOR_FORMAT_YUYV;
if (self->interlaced_flag)
{
-- 编译源YUY2补丁,现在我面临的问题是：
一个或两个摄像头显示绿色混乱图像后重新连接usb电缆可能修复或可能保持绿色,
使用v4l2src没有这样的问题。
gst-launch-1.0 -e nvv4l2camerasrc device=/dev/video1 !
'video/x-raw(memory:NVMM),format=YUY2,width=1920,height=1080,
framerate=30/1' ! nvvidconv ! 'video/x-raw(memory:NVMM)' ! nv3dsink
我的应用程序功有交换视频源的功能。
我送EOS到pipelien并等待它完成,然后我用交换的源启动一个全新的管道，但是在将新创建的管道状态设置为READY之后，gstreamer崩溃并产生以下输出。
我想提到的是，这种相同的方法被用于以前刷的镜像是没有问题的
0:00:03.877650773 19473 0xaf02cf0 WARN v4l2 gstv4l2object.c:2375:gst_v4l2_object_add_interlace_mode:0xabc1400 Failed to determine interlace mode
0:00:03.877743286 19473 0xaf02cf0 WARN v4l2 gstv4l2object.c:2375:gst_v4l2_object_add_interlace_mode:0xabc1400 Failed to determine interlace mode
0:00:03.877803766 19473 0xaf02cf0 WARN v4l2 gstv4l2object.c:2375:gst_v4l2_object_add_interlace_mode:0xabc1400 Failed to determine interlace mode
0:00:03.877909238 19473 0xaf02cf0 WARN v4l2 gstv4l2object.c:4434:gst_v4l2_object_probe_caps:<nvv4l2h265enc1:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1我认为问题的根源是在nvcompositor
recording_tee. ! queue ! rtph265pay config-interval=10 pt=96 !
udpsink host=127.0.01 port=5000然后去掉第二部分
recording_tee. ! queue ! splitmuxsink location=/mnt/storage/%02d.mp4 max-size-time=9000000000000
max-files=96 async-finalize=true然而，当我试图更改重新运行管道时，它失败了，只排除了整个comp.(最后3行)定义的帮助。除此之外，当我使用nvv4l2h265enc运行管道时，我经常看到以下消息
0:00:03.209421132 20591 0x7f38003850 FIXME h265parse gsth265parse.c:1798:gst_h265_parse_parse_frame:
Implement timestamp/duration interpolation based on SEI message这是我最后的pipeline
v4l2src io-mode=2 device=/dev/video0 name=source_1 ! video/x-raw,format=YUY2,width=1920,height=1080,framerate=30/1 ! nvvidconv ! video/x-raw(memory:NVMM) ! tee name=tee_source_1
v4l2src io-mode=2 device=/dev/video1 name=source_2 ! video/x-raw,format=YUY2,width=1920,height=1080,framerate=30/1 ! nvvidconv ! video/x-raw(memory:NVMM),width=640,height=360 ! tee name=tee_source_2
nvcompositor name=comp sink_0::width=1920 sink_0::height=1080 sink_1::xpos=1280 sink_1::width=640 sink_1::height=360
tee_source_1. ! queue ! comp.
tee_source_2. ! queue ! comp.
tee_source_1. ! queue ! nvvidconv ! video/x-raw,format=BGRx ! appsink drop=true emit-signals=true sync=false name=sink_primary
tee_source_2. ! queue ! nvvidconv ! video/x-raw,format=BGRx ! appsink drop=true emit-signals=true sync=false name=sink_secondary
comp. ! video/x-raw(memory:NVMM) ! nvvidconv ! video/x-raw(memory:NVMM),format=NV12 ! nvv4l2h265enc bitrate=8000000 ! h265parse ! tee name=recording_tee
recording_tee. ! queue ! rtph265pay config-interval=10 pt=96 ! udpsink host=127.0.0.1 port=5000
recording_tee. ! queue ! splitmuxsink location=/mnt/storage/2021-04-07/021642561645/%02d.mp4 max-size-time=9000000000000 max-files=964. 分析相机
从截图来看，颜色看起来不太对。
（1）检测相机格式
相机是YUY2格式的吗?
似乎是在UYVY或其他格式。
（2）相机类型
USB摄像头还是YUV传感器
使用DFG/HDSDI - SDI-to-USB 3.0 converter连接相机
（3）相机参数
v4l2-ctl -d /dev/video0 --list-formats-ext
Index : 1
Type : Video Capture
Pixel Format: 'YUYV'
Name : YUYV 4:2:2
Size: Discrete 1920x1080
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.033s (29.970 fps)
Interval: Discrete 0.040s (25.000 fps)
Interval: Discrete 0.042s (24.000 fps)
Interval: Discrete 0.042s (23.940 fps)
Size: Discrete 1920x540
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.017s (59.940 fps)
Interval: Discrete 0.020s (50.000 fps)
Size: Discrete 1280x720
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.017s (59.940 fps)
Interval: Discrete 0.020s (50.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.033s (29.970 fps)
Interval: Discrete 0.040s (25.000 fps)
Interval: Discrete 0.042s (24.000 fps)
Interval: Discrete 0.042s (23.940 fps)在JP4.5(r32.5)，使用nvvidconv和nvcompositor开源。
下载这个包并进一步调试:
https://developer.nvidia.com/embedded/l4t/r32_release_v5.1/r32_release_v5.1/sources/t186/public_sources.tbz2
经过进一步的调试，确定nvv4l2h265enc组件是错误的。
准备了一个简单的python应用程序，这是在我的主要应用程序中使用
import functools
import sys
import gi
from PyQt5 import QtWidgets
gi.require_version('Gst', '1.0')
from gi.repository import GObject, Gst
class MainWindow(QtWidgets.QWidget):
def __init__(self, parent=None):
super(MainWindow, self).__init__(parent)
self.setWindowTitle('QHBoxLayout')
layout = QtWidgets.QHBoxLayout()
button1 = QtWidgets.QPushButton('/dev/viode0')
button1.clicked.connect(functools.partial(self.create_pipeline, "/dev/video0"))
layout.addWidget(button1)
button2 = QtWidgets.QPushButton('/dev/video1')
button2.clicked.connect(functools.partial(self.create_pipeline, "/dev/video1"))
layout.addWidget(button2)
button3 = QtWidgets.QPushButton('Stop pipeline')
button3.clicked.connect(self.stop_pipeline)
layout.addWidget(button3)
self.setLayout(layout)
self.pipeline = None
self.bus = None
self.width = 1920
self.height = 1080
self.width_descaled = 640
self.height_descaled = 360
def create_pipeline(self, source):
self.stop_pipeline()
pipeline_str = \
f"v4l2src io-mode=2 device={source} name=source_1 ! video/x-raw,format=YUY2,width=1920,height=1080,framerate=30/1 ! nvvidconv ! video/x-raw(memory:NVMM) ! tee name=tee_source_1 " \
f"nvcompositor name=comp sink_0::width={self.width} sink_0::height={self.height} sink_1::xpos={self.width - self.width_descaled} sink_1::width={self.width_descaled} sink_1::height={self.height_descaled} " \
"tee_source_1. ! queue ! comp. " \
"tee_source_1. ! queue ! comp. " \
f"tee_source_1. ! queue ! nvvidconv ! video/x-raw(memory:NVMM),format=NV12 ! fakesink name=sink_primary " \
f"tee_source_1. ! queue ! nvvidconv ! video/x-raw(memory:NVMM),format=NV12,width={self.width_descaled},height={self.height_descaled} ! fakesink name=sink_secondary " \
"comp. ! video/x-raw(memory:NVMM) ! nvvidconv ! video/x-raw(memory:NVMM),format=NV12 ! tee name=comp_tee " \
"comp_tee. ! nv3dsink "
# FAULTY COMPONENT IS nvv4l2h265enc
pipeline_str += "comp_tee. ! nvv4l2h265enc maxperf-enable=1 bitrate=8000000"
print(pipeline_str)
self.pipeline = Gst.parse_launch(pipeline_str)
self.bus = self.pipeline.get_bus()
self.bus.add_signal_watch()
self.pipeline.set_state(Gst.State.READY)
self.pipeline.set_state(Gst.State.PLAYING)
def stop_pipeline(self):
timeout = 1
if self.pipeline is not None and self.pipeline.get_state(timeout).state == Gst.State.PLAYING:
Gst.Element.send_event(self.pipeline, Gst.Event.new_eos())
self.bus.timed_pop_filtered(Gst.CLOCK_TIME_NONE, Gst.MessageType.EOS)
self.pipeline.set_state(Gst.State.NULL)
self.pipline = None
self.bus = None
print("Pipeline stopped")
GObject.threads_init()
Gst.init(sys.argv)
Gst.debug_set_active(False)
Gst.debug_set_default_threshold(3)
app = QtWidgets.QApplication(sys.argv)
window = MainWindow()
window.show()
sys.exit(app.exec_())
声明：本文内容由易百纳平台入驻作者撰写，文章观点仅代表作者本人，不代表易百纳立场。如有内容侵权或者其他问题，请联系本站进行删除。
红包
94
8
评论
打赏
- 分享
- 举报
微信扫码分享
QQ好友
评论
0个
手气红包
暂无数据相关专栏
-
浏览量：10834次2021-05-06 16:22:01
-
2021-07-30 17:04:37
-
2021-07-30 17:20:54
-
浏览量：8575次2021-05-14 14:43:43
-
浏览量：6218次2021-07-09 15:17:28
-
浏览量：9200次2021-05-19 17:32:00
-
浏览量：7400次2021-05-11 17:51:48
-
浏览量：9445次2021-05-26 14:17:15
-
浏览量：9381次2021-04-27 17:56:41
-
浏览量：10184次2021-06-08 17:32:00
-
浏览量：5880次2021-05-28 13:52:17
-
浏览量：5027次2021-07-29 11:53:53
-
浏览量：7071次2021-05-11 17:04:57
-
浏览量：5237次2021-05-10 17:48:42
-
浏览量：5194次2021-07-09 15:23:33
-
浏览量：5259次2021-07-09 15:20:08
-
浏览量：7075次2021-05-20 16:37:42
-
浏览量：7889次2021-05-25 15:32:16
-
浏览量：9378次2021-05-25 17:31:40
关于作者
free-jdx
暂无个性签名~
原创178
阅读117.3w
收藏13
点赞35
评论25
切换马甲
上一页
下一页
打赏用户
共 1 位
我要创作
分享技术经验，可获取创作收益
分类专栏
-
178篇
热门专栏
- 所有努力只为给你更多欢乐！NGF次日精彩汇总
- Arm 现已支持 RTX！全球广泛应用的 CPU 架构融合实时光线追踪、DLSS
- 初创加速 | NVIDIA GPU加速AI智能数据处理，助力数字化医、药、险联动
- nvidia xavier平台无PD控制器USB接口调试
- 让战斗更爽快！《泰坦天降：远征》全新地图预览
- TITAN BLACK梦幻作品！GeForce Garage交叉书桌打造日志
- nvidia xavier nx平台使用system.img编写分区APP出错调试
- NVIDIA GeForce GTX TITAN Z资深粉丝评测
- Xavier-Nx模块上使GPIO7和GPIO14为PWM8和PWM7
- nvidia nx平台 Gstreamer tcpserverink延迟2-3秒问题调试1
置顶时间设置
结束时间
删除原因
-
广告/SPAM
-
恶意灌水
-
违规内容
-
文不对题
-
重复发帖
打赏作者
free-jdx
您的支持将鼓励我继续创作！
打赏金额：
￥1
￥5
￥10
￥50
￥100
支付方式：
微信支付
举报反馈
举报类型
- 内容涉黄/赌/毒
- 内容侵权/抄袭
- 政治相关
- 涉嫌广告
- 侮辱谩骂
- 其他
详细说明
审核成功
发布时间设置
发布时间：
请选择发布时间设置
是否关联周任务-专栏模块
审核失败
失败原因
请选择失败原因
备注
请输入备注
关注公众号
联系我们
社区问题咨询：Ebaina-CN
定制需求咨询：xxqk158820
社区问题咨询：Ebaina-CN
定制需求咨询：xxqk158820
回顶部
