Rockchip Linux4.4 Camera Trouble Shooting

:::info 文档信息

文档标识：RK-PC-YF-331　·　发布版本：V1.0.1　·　发布日期：2020-11-13　·　页数：18　·　语言：英文

原始文档：Rockchip_Trouble_Shooting_Linux4.4_Camera_EN.pdf

:::

章节大纲

• Rockchip Linux4.4 Camera Trouble Shooting（P1）
  • Whether the Sensor is Lighted up（P4）
    • Sensor ID Cannot be Recognized, I2C Communication Fails（P4）
      • What is 7-bits Address（P4）
      • The 24M mclk and vdd Power Supply Cannot be Detected After Power on（P4）
      • The 24M mclk Still Cannot be Detected（P4）
      • There is a mclk, But the Voltage Amplitude is Wrong and Different from Actual Power Domain（P4）
      • Check Whether the Power-on Sequence of Sensor Meets the Requirements（P5）
    • What are the Default Values of exp_def, hts_def, and vts_def in Sensor Driver（P5）
    • What Should be link_freq and pixel_rate Values（P6）
    • How to Make Sure a Sensor is Lighted Up（P6）
    • Sensor AVL List（P6）
  • MIPI/ISP Abnormal（P7）
    • What Parameters Need to be Set in MIPI（P7）
    • No Frame Data is Received, and No Error is Reported from ISP/MIPI（P7）
    • MIPI Error（P7）
      • The Detailed MIPI Error Message Table（P7）
      • How to Deal with SOT/SOT_SYNC Errors（P9）
      • How to Deal with CRC/CheckSum(CS), ECC/ECC1/ECC2 Errors（P9）
      • How to Deal with ERR_PROTOCOL/ERR_F_BNDRY Errors（P9）
      • Receive Frames Normally, but MIPI Errors Appear Occasionally（P9）
      • Report Many MIPI Errors or Even Crashes（P10）
      • How to Deal with ISP PIC_SIZE_ERROR（P10）
  • About Capturing Images（P10）
    • Ways to Capture Pictures（P11）
    • The Color of the Captured Image is Wrong, and the Brightness is Obviously Darker or Lighter（P11）
    • What is ISP Topology and How to Use media-ctl Command（P11）
      • How does One ISP Connect to Multiple Sensors（P12）
    • Whether Captured RAW Images are Exactly the Same as the Original Images（P12）
    • How does ISP Output two Channels (MP, SP) Simultaneously（P12）
    • Does ISP Have Amplify Function（P12）
    • Does ISP Have Rotation Function（P12）
    • How to Capture a Gray Image（P13）
    • Which Formats are Supported by RGB Images（P13）
    • How to Quickly Preview a Board Without Screen（P13）
    • How to Distinguish SP and MP（P13）
  • About 3A（P14）
    • How to Confirm the Version of camera_engine_rkisp（P14）
      • How to Confirm the Version Number of rkisp Kernel Driver Required by camera_engine_rkisp（P15）
    • How to Upgrade camera_engine_rkisp（P15）
    • How to Confirm Whether 3A is Working Properly（P16）
      • Did not Found the rkisp_3A_server Process（P16）
      • How to Start rkisp_3A_server（P16）
      • How to Make Sure the Name and Path of Sensor IQ Configuration File (xml)（P16）
    • How to Manually Expose（P17）
    • How to Open librkisp Log（P17）
  • Application Development（P17）
    • C Language Reference Demo（P17）
    • What is DMA Buffer and What are the Benefits（P18）

在线预览

若预览无法显示，请直接下载 PDF 后用本地阅读器打开。

---

1. Whether the Sensor is Lighted up

1.1 Sensor ID Cannot be Recognized, I2C Communication Fails

If sensor ID is not recognized, it has nothing to do with RKISP or RKCIF, but the sensor power-on sequence does not meet the requirements.

Please check the following items in orders:

1. Is the 7-bits i2c slave id of the sensor correct? Is it written as 8-bits by mistake? 2. Whether the 24M mclk has output and whether the voltage amplitude is correct? 3. Whether the power-on sequence of the sensor meets the requirements, including avdd, dovdd, dvdd, power down, reset, etc.

1.1.1 What is 7-bits Address

The least significant bit (LSB) of the 8bits represents R/W, and the higher 7bits is i2c slave id we needed.

1.1.2 The 24M mclk and vdd Power Supply Cannot be Detected After Power on

In the implementation of sensor driver, mclk and power are generally turned on only when needed, so mclk and power are turned off by default after booting.

When debugging, you can comment out the implementation of the power_off() function in driver, so that it will not be powered off and is convenient for measurement.

1.1.3 The 24M mclk Still Cannot be Detected

When using an oscilloscope, check whether the bandwidth of the oscilloscope is sufficient. A bandwidth of at least 48M is recommended.

1. When sensor does not open mclk correctly, please refer to the operation of mclk in

drivers/media/i2c/ov5695.c .

2. When the gpio is occupied by other modules, there will be some tips in kernel log. You can also use io command to check whether the pin-ctrl register setting is correct.

1.1.4 There is a mclk, But the Voltage Amplitude is Wrong and Different from Actual

Power Domain

This is due to a misconfiguration of io-domains. Generally, the voltage of io-domains is 1.8v or 3.3v, depending on your schematic design. Check the schematic diagram and modify io-domains accordingly, such as:

1 &io domains { _ 2 status = "okay"; 3 4 vccio1-supply = <&vcc1v8 soc>; _ 5 vccio2-supply = <&vccio sd>; _ 6 vccio3-supply = <&vcc1v8 dvp>; _ 7 vccio4-supply = <&vcc 3v0>; _ 8 vccio5-supply = <&vcc 3v0>; _ 9 };
1 2 3 4 5 6 7 8 9	&io domains { _ status = "okay"; vccio1-supply = <&vcc1v8 soc>; _ vccio2-supply = <&vccio sd>; _ vccio3-supply = <&vcc1v8 dvp>; _ vccio4-supply = <&vcc 3v0>; _ vccio5-supply = <&vcc 3v0>; _ };
In the

detected mclk should also be 1.8v.

During the boot process, io_domains may be initialized later than sensor. When sensor id reading fails, try to retry several times, as follows:

1 diff --git a/drivers/media/i2c/ov7251.c b/drivers/media/i2c/ov7251.c 2 index e0c608f..7842812 100644 3 --- a/drivers/media/i2c/ov7251.c 4 +++ b/drivers/media/i2c/ov7251.c 5 @@ -961,7 +961,8 @@ static int ov7251 check sensor id(struct ov7251 *ov7251, _ _ _ 6 OV7251 REG VALUE 16BIT, &id); _ _ _ 7 if (id != CHIP ID) { _ 8 dev err(dev, "Unexpected sensor id(%06x), ret(%d)\n", id, _ ret); 9 - return -ENODEV; 10 + return -EPROBE DEFER; _ 11 }
1 2 3 4 5 6 7 8 9 10 11	diff --git a/drivers/media/i2c/ov7251.c b/drivers/media/i2c/ov7251.c index e0c608f..7842812 100644 --- a/drivers/media/i2c/ov7251.c +++ b/drivers/media/i2c/ov7251.c @@ -961,7 +961,8 @@ static int ov7251 check sensor id(struct ov7251 *ov7251, _ _ _ OV7251 REG VALUE 16BIT, &id); _ _ _ if (id != CHIP ID) { _ dev err(dev, "Unexpected sensor id(%06x), ret(%d)\n", id, _ ret); - return -ENODEV; + return -EPROBE DEFER; _ }
If io d

default value is different from the actual hardware power domain, the voltage of mclk will not meet expectations. Returning -EPROBE_DEFER at this time will let sensor try probe again in the later boot process.

1.1.5 Check Whether the Power-on Sequence of Sensor Meets the Requirements

The datasheet of a sensor generally describes the power-on sequence and interval requirements of each power supply in details. Please check if it meets the requirements by an oscilloscope.

There is no time sequence requirement for power supply vdd of some sensors, such as ov5695, which may use regulator_bulk to manage the power supply in its driver; but some sensors have sequential requirements, such as ov2685.c, which uses multiple regulators to control separately in driver. such as avdd_regulator , dovdd_regulator . Please choose accordingly.

1.2 What are the Default Values of exp_def , hts_def , and

vts_def in Sensor Driver

If you have the contact information of the sensor vendor, please contact them to get. Otherwise, please find the corresponding register from the datasheet and find the value configured during initialization from the register list. Take ov2685.c as an example:

1 #define OV2685 REG VTS 0x380e _ _ 2 3 ... 4 5 {0x380e, 0x05}, 6 {0x380f, 0x0e}, 7 8 ... 9 10 .vts def = 0x050e, _
1 2 3 4 5 6 7 8 9 10	#define OV2685 REG VTS 0x380e _ _ ... {0x380e, 0x05}, {0x380f, 0x0e}, ... .vts def = 0x050e, _
The 0x

If you don't like application to adjust exposure and frame rate, exp, hts, vts are not needed . Generally, a sensor in RAW format requires these three parameters.

1.3 What Should be link_freq and pixel_rate Values

The link_freq refers to the actual frequency of MIPI clk. Note that it is not 24M mclk, but MIPI dn/dp clk. Check from the vendor first, or find whether the datasheet has relevant parameters.

Generally, the actual value of link_freq will not be less than the calculation result of the following formula with unit is (Hz)

1 width * height * fps * bits per pixel / lanes / 2 _ _
1	width * height * fps * bits per pixel / lanes / 2 _ _

If you really don’t know the actual value of link_freq , you can measure by an oscilloscope.

The pixel_rate refers to the number of pixels transmitted per second. After link_freq is determined, the following formula can be used to calculate:

1 link freq * 2 * lanes / bits per pixel _ _ _
1	link freq * 2 * lanes / bits per pixel _ _ _

1.4 How to Make Sure a Sensor is Lighted Up

Firstly, the sensor id can be recognized, that is, there should be no abnormalities in the reading and writing of i2c. At this time, the detailed information of the sensor such as the name and resolution should be able to checked by

media-ctl -p -d /dev/media0 .

Secondly, when upper layer captures pictures, MIPI must be able to output data without reporting MIPI/ISP errors, and application layer can receive frames.

1.5 Sensor AVL List

RGB sensor AVL is located at https://redmine.rockchip.com.cn/projects/rockchip_camera_module_support_list/c amera. The detailed information of sensor modules are shown in the support list.

If it is other non-RGB sensor, such as YUV sensor, you can directly find the drivers/media/i2c/ directory of the kernel source code. The drivers have all been tuned by Rockchip.

2. MIPI/ISP Abnormal

In the early stage of sensor tunning, the following problems maybe frequently encountered:

1. No frame data is received, and no error is reported by ISP/MIPI 2. Keep printing MIPI errors log 3. ISP reports PIC_SIZE_ERROR 4. MIPI errors occasionally 5. MIPI keeps reporting errors until it crashes

2.1 What Parameters Need to be Set in MIPI

There are 4 parameters need to been set in MIPI communication between sensor and ISP, please confirm the correctness of the 4 MIPI parameters.

1. Resolution of sensor output 2. The image format output from sensor is YUV or RGB RAW, 8-bits, 10-bits, or 12-bits 3. The ink_freq output from sensor's MIPI 4. If there are several MIPI lanes in sensor, please make sure they are configured correctly in dts

2.2 No Frame Data is Received, and No Error is Reported from

ISP/MIPI

1. Confirm whether there are any errors about MIPI in kernel log. For example, check if there are any errors

by  dmesg | grep MIPI .

2. Confirm whether there is a sensor's i2c read and write failure in kernel log. If sensor fails to configure registers, the sensor may not be initialized correctly and enable output. 3. Measure whether there is information output from clk and data lines of MIPI. If not, it is recommended to analyze sensor initialization and hardware. 4. There is MIPI signal output in measure, but no error is reported and no data can be received:

Please check according to the Section 2.1 What Parameters Need to be Set in MIPI again Please confirm that there is no error in I2C communication, and the sensor register initialization list have all been written to the sensor In sensor driver, the last to enable MIPI output is s_stream() . Please make sure that before this function, especially s_power() , there is no MIPI signal output. The reason is MIPI controller is not actually ready to receive data before s_stream() . If data is output before s_stream() , it may cause the loss of MIPI protocol header SOT signal. Switch the clock lane on camera sensor side from continue mode to no continues.

2.3 MIPI Error

2.3.1 The Detailed MIPI Error Message Table

For RK3288/RK3399/RK3368, the error message table is as follows:

Error bit（Bit）	Abbreviation	Description
25	ADD DATA OVFLW _ _	additional data fifo overflow occurred
24	FRAME END _	A frame is received normally, not an error
23	ERR CS _	checksum error
22	ERR ECC1 _	1-bit ecc error
21	ERR ECC2 _	2-bit ecc error
20	ERR PROTOCOL _	packet start detected within current packet
19:16	ERR CONTROL _	PPI interface control error occured, one bit per lane
15:12	ERR EOT SYNC _ _	MIPI EOT(End Of Transmission) sync, one bit per lane
11:8	ERR SOT SYNC _ _	MIPI SOT(Start Of Transmission) sync, one bit per lane
7:4	ERR SOT _	MIPI SOT(Start Of Transmission), one bit per lane
3:0	SYNC FIFO OVFLW _ _	synchronization fifo overflow occurred, one bit per lane

For RK3326/PX30/RK1808, the three error message tables are as follows:

ERR1 Error Bit（Bit）	Abbreviation	Description
28	ERR ECC _	ECC ERROR
27:24	ERR CRC _	CRC ERROR
23:20	ERR FRAME DATA _ _	Frame transmission is complete, but at least one CRC error is included
19:16	ERR F SEQ _ _	Frame Number is not continuous will not meet expectations
15:12	ERR F BNDRY _ _	Frame start and Frame end do not match
11:8	ERR SOT SYNC _ _	MIPI PHY SOT(Start Of Transmission) sync error
7:4	ERR EOT SYNC _ _	MIPI PHY EOT(End Of Transmission) sync error

ERR2 Error Bit（Bit）	Abbreviation	Description
19:16	ERR CONTROL _
15:12	ERR ID _
11:8	ERR ECC CORRECTED _ _
7:4	ERR SOTHS _	PHY SOTHS error
3:0	ERR ESC _	PHY ESC error

Please read the following chapters for analysis of frequently encountered problems.

2.3.2 How to Deal with SOT/SOT_SYNC Errors

The SOT signal needs to comply with MIPI_D-PHY_Specification. If you need in-depth analysis, please search pdf documents directly from Internet, and it is suggested to focus on the following items:

1. High-Speed Data Transmission 2. Start-of-Transmission Sequence 3. HS Data Transmission Burst 4. High-Speed Clock Transmission 5. Global Operation Timing Parameters

But generally speaking, if the sensor has tuned on other platforms, there will be little possibility of not complying with MIPI protocol. It is recommended to customers that:

1. Please first confirm to sensor vendor whether the sensor has actually successfully transmit data on MIPI interface 2. Reconfirm whether link_freq is correct. Because the Ths-settle in SOT sequence needs to be configured correctly at MIPI receiving side, link_freq is so important. 3. If multiple lanes are used, see if the sensor vendor can modify it to 1 lane for transmission.

2.3.3 How to Deal with CRC/CheckSum(CS), ECC/ECC1/ECC2 Errors

When ECC error and CS check error occurred, it means that data was incomplete during transmission. It is suggested that:

1. Check hardware signals firstly 2. If multiple lanes are used, check if the sensor vendor can modify it to 1 lane for transmission. Because multiple lanes are not synchronized well, ECC errors may also occur.

2.3.4 How to Deal with ERR_PROTOCOL/ERR_F_BNDRY Errors

This error indicates that the expected EOT/SOT was not received. And SOT and EOT should appear in pairs. It is recommended to check the waveform.

2.3.5 Receive Frames Normally, but MIPI Errors Appear Occasionally

If it is a MIPI error, please refer to the above error description. It is recommended to analyze the signal related errors from hardware.

In particular, if MIPI error occurs only at the beginning of capturing images, it is possible that there is MIPI signal output during sensor power-on but it does not conform to the protocol, so the error is reported. In this case, you can try to modify as follows:

1. Put the initialization of the complete sensor register into s_power() . Because MIPI receiver has not yet started receiving data at this time, all data will be ignored. 2. At the end of the s_power() function, turning off the output of sensor, in other words, to call

stop_stream()

3. In start_stream() and stop_stream() , output of MIPI is turned on or off only.

2.3.6 Report Many MIPI Errors or Even Crashes

It may be a worse situation mentioned in section 2.3.5 Receive Frames Normally, but MIPI Errors Appear Occasionally.

The reason for this is that MIPI signal does not meet the requirement, and some errors at MIPI receiving side are voltage interruptions, which leads to an interruption storm and eventually crashes.

You can try to check whether it will take effect according to section 2.3.5 Receive Frames Normally, but MIPI Errors Appear Occasionally.

2.3.7 How to Deal with ISP PIC_SIZE_ERROR

Picture size error is an ISP level error, which indicates that the expected number of rows and columns have not been received. So check resolution size from all levels.

If the previous level (that is MIPI) reported an error, the MIPI error should be resolved first.

Please check the following items:

1. Is DDR frequency too small? This error can also occur when DDR frequency is too low and the response speed is insufficient. Try to fix DDR frequency to the highest frequency to see if it will go wrong: echo

performance > /sys/class/devfreq/dmc/governor

2. In the whole ISP link, is there a case that the resolution of the latter stage is greater than that of the previous stage. You can use media-ctl -p -d /dev/media0 to check the topology.The resolution should satisfy Sensor == MIPI_DPHY >= isp_sd input >= isp_sd output. If you haven't modified it manually, it should meet this condition by default. 3. Is the output resolution of sensor correct? Try to force the resolution to be smaller in the driver code. For example, the default resolution in ov7251.c is 640x480:

1 static const struct ov7251 mode supported modes[] = { _ _ 2 { 3 .width = 640, 4 .height = 480,
1 2 3 4	static const struct ov7251 mode supported modes[] = { _ _ { .width = 640, .height = 480,

Change the width and height to be smaller, such as 320x240, the register configuration does not need to be changed, in order to confirm whether the configuration size of the sensor will exceed the actual output size.

1 static const struct ov7251 mode supported modes[] = { _ _ 2 { 3 .width = 320, 4 .height = 240,
1 2 3 4	static const struct ov7251 mode supported modes[] = { _ _ { .width = 320, .height = 240,

3. About Capturing Images

This chapter will going to introduce common problems related to capturing images.

3.1 Ways to Capture Pictures

RKISP and RKCIF drivers support v4l2 interface, you can use the following ways to capture images:

1. Use the v4l2-ctl tool in the v4l-utils package to capture images. In the tuning process, it is firstly recommended to use this tool to check whether pictures can successfully output.The v4l2-ctl capture pictures and saved them as a file, but it cannot parse images and display them. If you need parse, mplayer can be used in Ubuntu/Debian, and 7yuv, etc. can be used in Windows.For detailed instructions of v4l2-ctl and mplayer tools, please refer to "Rockchip_Developer_Guide_Linux_Camera_CN.pdf".The v4l2-ctl also comes with detailed v4l2-ctl --help document. 2. The Linux SDK provided by RK contains qcamera application app.Open the camera app on the desktop directly, select the video device and preview.The app is a demo based on Qt, for reference only. In customer projects, it is recommended to develop apps by yourselves. 3. Use v4l2src plugin of gstreamer to get images from /dev/video devices and display them on the screen. The Linux SDK provided by RK will contain some scripts under the directory /rockchip_test/camera/, please refer to it first.

Note: RK has provided multiple versions of gstreamer isp plugin, such as rkisp, rkv4l2src, which are no longer supported. Please use the v4l2src plugin that comes with gstreamer directly. There are two main reasons:

3A no longer needs to be adjusted in rkisp or rkv4l2src. Please refer to Chapter 4 About 3A for details v4l2 interface driven by rkisp is more standardized

4. Open source tools such as vlc are used in Debian system

After installing vlc through apt, you can use the following command to display the camera image on the screen:

1 vlc v4l2:///dev/video1:width=640:height=480
1	vlc v4l2:///dev/video1:width=640:height=480

Note that the permission of the video user group or root super user permission is required.

3.2 The Color of the Captured Image is Wrong, and the Brightness is

Obviously Darker or Lighter

According to types of sensors:

1. If sensor is output of RAW RGB, such as RGGB, BGGR, etc. It needs 3A to run normally. Please refer to Chapter 4 About 3A. When 3A is running normally, please check again whether the format used when parsing/displaying images is correct and whether the uv component is reversed. 2. If sensor is yuv output, or RGB such as RGB565, RGB888, at this time ISP is in bypass state,

If the color is wrong, please confirm whether the output format of sensor is misconfigured and whether the uv component is reversed. When you confirmed that they are correct, it is recommended to contact the sensor vendor If the brightness is obviously wrong, please contact the sensor vendor

3.3 What is ISP Topology and How to Use media-ctl Command

RKISP or RKCIF can be connected to multiple sensors, time-sharing multiplexing; at the same time, RKISP also has a multi-level cutting function. Therefore, each node is connected by topology, and the parameters can be configured separately through media-ctl. About the use of media-ctl, there is a more complete description in the "Rockchip_Developer_Guide_Linux_Camera_CN.pdf" document.

3.3.1 How does One ISP Connect to Multiple Sensors

Multiple sensors can be connected, but only time-sharing multiplexing. By configuring dts, after linking multiple sensors to MIPI DPHY, the sensors can be switched through media-ctl.

3.4 Whether Captured RAW Images are Exactly the Same as the

Original Images

When ISP capture sensor RAW images (such as RGGB, BGGR) in the bypass mode, 8bit alignment is required, and if it is less than 8bit, the low bit will be filled with 0, that is

If they are 8bit, 16bit original images, images captured by the application are the original, and not filled If they are 10bit, 12bit original images, the low bit of each pixel will be filled with 0 to 16bit

Only the video device corresponding to MP can output RAW images, SP cannot support RAW image output.

3.5 How does ISP Output two Channels (MP, SP) Simultaneously

RKISP has two outputs: SP and MP, that is, an image from sensor, SP and MP can cut and change format of the image separately, and output at the same time.

For the detailed video processing capabilities of SP and MP, please refer to "Rockchip_Developer_Guide_Linux_Camera_CN.pdf".

Only when SP and MP both output RGB or YUV, they can output at the same time. If MP outputs RAW images, then there will be no output from SP

3.6 Does ISP Have Amplify Function

This function is supported on the hardware, but it is not recommended to use it. And it is also turned off by default in the driver.

3.7 Does ISP Have Rotation Function

No. If you need the rotation function, it is recommended to:

If it is flip, mirror, first check if the sensor has this function, if so, use it directly. This is the most efficient If there is no sensor flip, mirror, RGA module is consideration, its code and demo are located in the external/linux-rga/ directory, and related documents are located in the docs/ directory

3.8 How to Capture a Gray Image

As long as ISP can output YUV or sensor output is Y8 gray image, application can always use V4L2_PIX_FMT_GREY (FourCC for GREY) format to capture images directly.

3.9 Which Formats are Supported by RGB Images

First of all, only SP can support RGB output, the format are: V4L2_PIX_FMT_XBGR32, V4L2_PIX_FMT_RGB565. The XBGR32 (corresponding FourCC is XR24) contains four components of R, G, B, and X, and X component is always 0. Does not support RGB888, that is, 24bit format output.

3.10 How to Quickly Preview a Board Without Screen

The external/uvc_app/ directory in the SDK provides the function of simulating the board into an uvc camera. Please refer to the introduction file and code in the directory. After the board is connected to a PC, an usb camera can be recognized and images can be previewed.

3.11 How to Distinguish SP and MP

The media-ctl -p -d /dev/media0 (if there are multiple media devices, /dev/media1, /dev/media2 can be tried, too) can be used to see topology, and list part of the output as follows:

1 # media-ctl -p -d /dev/media0 2 ... 3 - entity 2: rkisp1 mainpath (1 pad, 1 link) //Means the entity _ is MP(MainPath) 4 type Node subtype V4L flags 0 5 device node name /dev/video1 //The corresponding device node is/dev/video1 6 pad0: Sink 7 <- "rkisp1-isp-subdev":2 [ENABLED] 8 9 - entity 3: rkisp1 selfpath (1 pad, 1 link) //Means the entity _ is SP(SelfPath) 10 type Node subtype V4L flags 0 11 device node name /dev/video2 //The corresponding device node is/dev/video2 12 pad0: Sink 13 <- "rkisp1-isp-subdev":2 [ENABLED] 14 ...
1 2 3 4 5 6 7 8 9 10 11 12 13 14	# media-ctl -p -d /dev/media0 ... - entity 2: rkisp1 mainpath (1 pad, 1 link) //Means the entity _ is MP(MainPath) type Node subtype V4L flags 0 device node name /dev/video1 //The corresponding device node is/dev/video1 pad0: Sink <- "rkisp1-isp-subdev":2 [ENABLED] - entity 3: rkisp1 selfpath (1 pad, 1 link) //Means the entity _ is SP(SelfPath) type Node subtype V4L flags 0 device node name /dev/video2 //The corresponding device node is/dev/video2 pad0: Sink <- "rkisp1-isp-subdev":2 [ENABLED] ...
In a fe

1 # grep '' /sys/class/video4linux/video*/name 2 /sys/class/video4linux/video0/name:stream cif _ 3 /sys/class/video4linux/video1/name:rkisp1 mainpath # Corresponding MP _ node/dev/video1 4 /sys/class/video4linux/video2/name:rkisp1 selfpath # Corresponding SP _ Node/dev/video2 5 /sys/class/video4linux/video3/name:rkisp1 rawpath _ 6 /sys/class/video4linux/video4/name:rkisp1 dmapath _ 7 /sys/class/video4linux/video5/name:rkisp1-statistics 8 /sys/class/video4linux/video6/name:rkisp1-input-params

1 2 3 4 5 6 7 8

# grep '' /sys/class/video4linux/video*/name /sys/class/video4linux/video0/name:stream cif _ /sys/class/video4linux/video1/name:rkisp1 mainpath # Corresponding MP _ node/dev/video1 /sys/class/video4linux/video2/name:rkisp1 selfpath # Corresponding SP _ Node/dev/video2 /sys/class/video4linux/video3/name:rkisp1 rawpath _ /sys/class/video4linux/video4/name:rkisp1 dmapath _ /sys/class/video4linux/video5/name:rkisp1-statistics /sys/class/video4linux/video6/name:rkisp1-input-params

4. About 3A

If a sensor needs 3A tunning, like RAW BAYER RGB format of RGGB, BGGR ,etc. format output from sensor, then RKISP is required to provide image processing.

Depending on the version of camera_engine_rkisp, 3A processing methods are different. It is recommended to upgrade camera_engine_rkisp to the latest version as much as possible.

Please first confirm whether the module is in the support list

For those already in the support list, there will be a corresponding xml file in the external/camera_engine_rkisp/iqfiles/ directory Otherwise, please apply a module tuning application by business

4.1 How to Confirm the Version of camera_engine_rkisp

1. Check from the source code

1 # grep CONFIG CAM ENGINE LIB VERSION interface/rkisp dev manager.h _ _ _ _ _ _ 2 define CONFIG CAM ENGINE LIB VERSION "v2.2.0" # The output v2.2.0 _ _ _ _ is the version number of librkisp.so
1 2	# grep CONFIG CAM ENGINE LIB VERSION interface/rkisp dev manager.h _ _ _ _ _ _ define CONFIG CAM ENGINE LIB VERSION "v2.2.0" # The output v2.2.0 _ _ _ _ is the version number of librkisp.so

2. Check from running log

1 # persist camera engine log=0x4000 rkisp 3A server --mmedia=/dev/media1 | _ _ _ _ _ grep "CAM ENGINE LIB VERSION" 2 CAM ENGINE LIB VERSION IS v2.2.0 # The output v2.2.0 is the version number of librkisp.so 3
1 2 3	# persist camera engine log=0x4000 rkisp 3A server --mmedia=/dev/media1 | _ _ _ _ _ grep "CAM ENGINE LIB VERSION" CAM ENGINE LIB VERSION IS v2.2.0 # The output v2.2.0 is the version number of librkisp.so

If the version number is lower than v2.2.0, please consider upgrading to v2.2.0 or a newer version

4.1.1 How to Confirm the Version Number of rkisp Kernel Driver Required by

camera_engine_rkisp

The camera_engine_rkisp has requirements for kernel driver version, so you need to ensure that the rkisp driver is latest enough.

1. Check ISP driver version from the kernel source code

1 # grep RKISP1 DRIVER VERSION drivers/media/platform/rockchip/isp1/version.h _ _ 2 define RKISP1 DRIVER VERSION KERNEL VERSION(0, 1, 0x5) # The output v0.1.5 _ _ _ is the version number of the rkisp driver
1 2	# grep RKISP1 DRIVER VERSION drivers/media/platform/rockchip/isp1/version.h _ _ define RKISP1 DRIVER VERSION KERNEL VERSION(0, 1, 0x5) # The output v0.1.5 _ _ _ is the version number of the rkisp driver

2. Check ISP driver version from the kernel log

1 # dmesg | grep "rkisp1 driver version" 2 [ 0.867864] rkisp1 ff4a0000.rkisp1: rkisp1 driver version: v00.01.05 3
1 2 3	# dmesg | grep "rkisp1 driver version" [ 0.867864] rkisp1 ff4a0000.rkisp1: rkisp1 driver version: v00.01.05

4.2 How to Upgrade camera_engine_rkisp

There are three parts:

1. The body of camera_engine_rkisp

It is located in the external/camera_engine_rkisp directory of the SDK, it can be updated directly through

git or repo tools. You can only update the directory without affecting the directories in other SDKs.

2. Kernel is upgraded according to needs of camera_engine_rkisp

By checking git log in the external/camera_engine_rkisp directory, you can find the version number of the kernel rkisp driver. For example:

1 # git log 2 commit e456a50a5524792d64dac384604d4136a697deac 3 Author: ZhongYichong <zyc@rock-chips.com> 4 Date: Mon Jul 1 11:26:32 2019 +0800 5 6 librkisp: v2.2.0 7 8 (BY ZSQ: UPDATE v2.2.0 iq version: from v1.4.0 to v1.5.0) 9 10 3A lib version: 11 af: v0.2.17 12 awb: v0.0.e 13 aec: v0.0.e 14 iq version: v1.5.0 15 iq magic version code: 706729 16 17 matched rkisp1 driver version: 18 v0.1.5 # The needed kernel driver version is v0.1.5 19 20 Change-Id: I3d2adb949dadec259b9ba587a3e3b2770a1c155d 21 Signed-off-by: ZhongYichong <zyc@rock-chips.com> 22 Signed-off-by: Shunqian Zheng <zhengsq@rock-chips.com>

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

# git log commit e456a50a5524792d64dac384604d4136a697deac Author: ZhongYichong <zyc@rock-chips.com> Date: Mon Jul 1 11:26:32 2019 +0800 librkisp: v2.2.0 (BY ZSQ: UPDATE v2.2.0 iq version: from v1.4.0 to v1.5.0) 3A lib version: af: v0.2.17 awb: v0.0.e aec: v0.0.e iq version: v1.5.0 iq magic version code: 706729 matched rkisp1 driver version: v0.1.5 # The needed kernel driver version is v0.1.5 Change-Id: I3d2adb949dadec259b9ba587a3e3b2770a1c155d Signed-off-by: ZhongYichong <zyc@rock-chips.com> Signed-off-by: Shunqian Zheng <zhengsq@rock-chips.com>

3. The compilation script of camera_engine_rkisp in buildroot

It is located in the directory of buildroot/package/rockchip/camera_engine_rkisp. If it is not convenient to update the whole buildroot, you can update this directory alone.

4.3 How to Confirm Whether 3A is Working Properly

After confirming that camera_engine_rkisp is already v2.2.0 or later version. By capturing an image, check whether the color and exposure of the image are normal.

At the same time, by checking whether there is rkisp_3A_server process in the background, as follows:

1 # ps -ef | grep rkisp 3A server _ _ 2 706 root 9176 S /usr/bin/rkisp 3A server --mmedia=/dev/media1 _ _ 3 746 root 2408 S grep rkisp 3A server _ _ 4 # pidof rkisp 3A server _ _ 5 706
1 2 3 4 5	# ps -ef | grep rkisp 3A server _ _ 706 root 9176 S /usr/bin/rkisp 3A server --mmedia=/dev/media1 _ _ 746 root 2408 S grep rkisp 3A server _ _ # pidof rkisp 3A server _ _ 706

You can see that the process number 706 is rkisp_3A_server.

4.3.1 Did not Found the rkisp_3A_server Process

1. First confirm whether the /usr/bin/rkisp_3A_server executable file exists, if not, please check the camera_engine_rkisp version and build it. 2. Check whether there are rkisp_3A related errors in /var/log/syslog. If so, see what the detailed error is and whether the xml corresponding to the sensor module is not found or does not match. 3. Execute rkisp_3A_server --mmedia=/dev/media0 in a shell (if there are multiple /dev/media devices, select the one corresponding to /dev/video), and capture pictures from another shell, To get the error message corresponding to rkisp_3A_server

4.3.2 How to Start rkisp_3A_server

In the Linux SDK, rkisp_3A_server is started by the script /etc/init.d/S40rkisp_3A and executed in the background.

If the /etc/init.d/S40rkisp_3A file is not found, check the version of camera_engine_rkisp and the buildroot package compilation script.

4.3.3 How to Make Sure the Name and Path of Sensor IQ Configuration File (xml)

The sensor iq file consists of three parts,

1. Sensor Type, such as ov5695, imx327

2. Module Name, is defined in dts, for example, rk3326/px30 rk evb board, the name is "TongJu":

rockchip,camera-module-name = "TongJu";

3. Module Lens Name, defined in dts, such as the following "CHT842-MD":

rockchip,camera-module-lens-name = "CHT842-MD";

The iq file name in the above example is: ov5695_TongJu_CHT842-MD.xml, which is stored in the /etc/iqfiles/ directory. Note the case-sensitive.

4.4 How to Manually Expose

In the case of manual exposure, the rkisp_3A_server process must exit first. Then you can refer to the source code of rkisp_demo.cpp program or librkisp_api.so.

4.5 How to Open librkisp Log

By setting the environment variable persist_camera_engine_log, the corresponding bits are list as follows:

1 bits: 23-20 19-16 15-12 11-8 7-4 3-0 2 module: [xcore] [ISP] [AF] [AWB] [AEC] [NO] 3 4 0: error 5 1: warning 6 2: info 7 3: verbose 8 4: debug
1 2 3 4 5 6 7 8	bits: 23-20 19-16 15-12 11-8 7-4 3-0 module: [xcore] [ISP] [AF] [AWB] [AEC] [NO] 0: error 1: warning 2: info 3: verbose 4: debug
For e

1 # /etc/init.d/S40rkisp 3A stop _ 2 # export persist camera engine log=0x040400 _ _ _ 3 # /usr/bin/rkisp 3A server --mmedia=/dev/media0 _ _
1 2 3	# /etc/init.d/S40rkisp 3A stop _ # export persist camera engine log=0x040400 _ _ _ # /usr/bin/rkisp 3A server --mmedia=/dev/media0 _ _

5. Application Development

5.1 C Language Reference Demo

1. The Linux SDKs provided by RK include rkisp_demo tool and source code

rkisp_demo is a simple tool that can be used to capture images. Like v4l2-ctl tool, rkisp_demo can't display images either. It mainly provides source code for reference.

The source code is located in the external/camera_engine_rkisp/apps directory. If your code is out of date, please refer to the source code is located in the external/camera_engine_rkisp/tests directory.

2. The Linux SDKs provided by RK include rkisp_api.so dynamic link library and source code, you can modify base on it or directly use C language to develop programs

The source code is located in the external/camera_engine_rkisp/apps directory. If your code is out of date and cannot find the directory, please update.

5.2 What is DMA Buffer and What are the Benefits

DMA buffer is a part of memory allocated by driver. This buffer can be shared among multiple kernel modules to reduce memory copy. Especially for image processing, it can optimize performance and reduce DDR load.

For example, the camera image needs to be encoded by mpp:

DMA buffer shared

RKISP

MPP coding

RKISP can receive DMA buffer from other modules (such as MPP, RGA), and can also allocate memory and

export DMA buffer to other modules.

For details, please refer to the source code of librkisp_api.so library.