触摸屏研究工作

TSLIB

https://github.com/kergoth/tslib
user space，辅助读写/dev/input设备文件的库
使用前需export

make install后，会在/home/latelee/lib/tslib目录生成4个子目录：  
include lib etc bin  
生成的库位于lib中，该目录下还有一个子目录ts，它包含了许多校准用到的库(如input.so等)。etc下的ts.conf为配置文件，bin目录下为校准、测试工具(如校准的ts_calibrate，测试用的ts_test)。  
配置文件如下：  
# Late Lee 2011-03-31 for touchscreen test  
export TSLIB_ROOT=/usr/local/tslib  
export TSLIB_TSDEVICE=/dev/input/event0  
export TSLIB_CONFFILE=$TSLIB_ROOT/etc/ts.conf  
export TSLIB_PLUGINDIR=$TSLIB_ROOT/lib/ts  
export TSLIB_CALIBFILE=/etc/pointercal  
export TSLIB_CONSOLEDEVICE=none  
export TSLIB_FBDEVICE=/dev/fb0  
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$TSLIB_ROOT/lib  
(该文件的日期为上个月最后一天，离今已十余天，为了这个花费的时日可不少。)  
使用TSLIB_ROOT这个变量比较好，因为只要将编译生成后的tslib文件夹复制到某个位置，再修改这个变量即可，其它的不用修改。  
配置文件ts.conf内容如下：  
module_raw input  
module pthres pmin=1  
module variance delta=30  
module dejitter delta=100  
module linear  
module_raw有许多种，这里只使用input(即Linux的input子系统，设备文件名称为/dev/input/event0)，其它的删除掉。后面的几个module还没有深入了解，它们使用的库就在tslib/lib/ts中，最后三个模块的字面意思是“方差(滤波)”、“去抖动(去噪)”、“线性(坐标变换)”，对这些东西不了解，不发表意见。

https://github.com/kergoth/tslib/tree/master/plugins ^ 以上提到的modules的源码
...bin/ts_calibrate 校准
读取config文件：

// https://github.com/kergoth/tslib/blob/master/plugins/linear.c#L254  

if ((calfile = getenv("TSLIB_CALIBFILE")) == NULL)  
  calfile = TS_POINTERCAL;  

if (stat(calfile, &sbuf) == 0) {  
  pcal_fd = fopen(calfile, "r");  
  if (!pcal_fd) {  
    free(lin);  
    perror("fopen");  
    return NULL;  
  }

https://wiki.qt.io/I.MX-6
http://blog.csdn.net/xishuang_gongzi/article/details/49422879 tslib移植
12 错误处理
①./autogen.sh执行失败是因为缺少工具，通过sudo apt-get install autoconf automake lib>tool安装相应工具。
②
Couldnt open tslib config file: No such file or directory
ts_config: No such file or directory
原因：一般为环境变量 TSLIB_PLUGINDIR 设置问题。可以查看ts_config.c代码，打印出具体路径查看错误原>因
③打不开module input
Couldnt load module input
No raw modules loaded.
tsconfig: Success
原因分析：
查看ts.conf中module_raw的配置
编译选项需要增加 --enable_input=yes 否则没有input.so文件
④
ts open
tslib: Selected device is not a Linux input event device
原因分析：input-raw.c 函数需要根据底层驱动做适当修改。
触摸屏驱动没有提供ioctl接口，要删除代码中与ioctl有关的所有函数。
⑤Segmentation fault
解决：一般是因为配置文件ts.conf格式错误。参数前面不能有空格。
⑥
#ts_calibrate
ts_open: No such file or directory
可能是触摸屏驱动没有加载，找不到设备节点
http://stackoverflow.com/questions/28868963/qt-5-4-linux-touchscreen-input-with-tslib-on-raspberry-pi-failing-with-linuxfb-q A problem with qt

tslib 与 EVDEV 冲突

http://stackoverflow.com/questions/31580004/qt-5-3-2-tslib-wrong-touch-coordinates

missing this:

https://patchwork.openembedded.org/patch/50589/

实际项目

QT提供了不同的输入系统，以plugin的形式

/home/samba/sda3/freescale/3.14.52/yocto-L3.14.52/fb/tmp/work/cortexa9hf-vfp-neon-mx6qdl-poky-linux-gnueabi/qtbase/5.5.0+gitAUTOINC+c619d2daac-r0/git/src/plugins:

➜  plugins git:(c619d2d) ✗ tree generic 
generic  
├── evdevkeyboard  
│   ├── evdevkeyboard.json  
│   ├── evdevkeyboard.pro  
│   ├── main.cpp  
│   ├── Makefile  
│   └── README  
├── evdevmouse  
│   ├── evdevmouse.json  
│   ├── evdevmouse.pro  
│   ├── main.cpp  
│   ├── Makefile  
│   └── README  
├── evdevtablet  
│   ├── evdevtablet.json  
│   ├── evdevtablet.pro  
│   ├── main.cpp  
│   ├── Makefile  
│   └── README  
├── evdevtouch  
│   ├── 70-qtouchscreen.rules  
│   ├── evdevtouch.json  
│   ├── evdevtouch.pro  
│   ├── evdevtouch.pro.user  
│   ├── main.cpp  
│   ├── Makefile  
│   └── README  
├── generic.pro  
├── libinput  
│   ├── libinput.json  
│   ├── libinput.pro  
│   ├── main.cpp  
│   ├── Makefile  
│   └── README  
├── Makefile  
├── tslib  
│   ├── main.cpp  
│   ├── Makefile  
│   ├── tslib.json  
│   └── tslib.pro  
└── tuiotouch  
    ├── main.cpp  
    ├── Makefile  
    ├── qoscbundle.cpp  
    ├── qoscbundle_p.h  
    ├── qoscmessage.cpp  
    ├── qoscmessage_p.h  
    ├── qtuiocursor_p.h  
    ├── qtuiohandler.cpp  
    ├── qtuiohandler_p.h  
    ├── qtuio_p.h  
    ├── README.md  
    ├── tuiotouch.json  
    └── tuiotouch.pro

qt-everywhere-opensource-src-5.5.0/qtbase/src/plugins:

pnck@pnix:~/sourcecodes/qt/qt-everywhere-opensource-src-5.5.0/qtbase/src/plugins$ tree generic/  
generic/  
├── evdevkeyboard  
│   ├── evdevkeyboard.json  
│   ├── evdevkeyboard.pro  
│   ├── main.cpp  
│   ├── Makefile  
│   └── README  
├── evdevmouse  
│   ├── evdevmouse.json  
│   ├── evdevmouse.pro  
│   ├── main.cpp  
│   ├── Makefile  
│   └── README  
├── evdevtablet  
│   ├── evdevtablet.json  
│   ├── evdevtablet.pro  
│   ├── main.cpp  
│   ├── Makefile  
│   └── README  
├── evdevtouch  
│   ├── 70-qtouchscreen.rules  
│   ├── evdevtouch.json  
│   ├── evdevtouch.pro  
│   ├── main.cpp  
│   ├── Makefile  
│   └── README  
├── generic.pro  
├── libinput  
│   ├── libinput.json  
│   ├── libinput.pro  
│   ├── main.cpp  
│   └── README  
├── Makefile  
├── sub-evdevkeyboard-make_first  
├── sub-evdevmouse-make_first  
├── sub-evdevtablet-make_first  
├── sub-evdevtouch-make_first  
├── sub-tuiotouch-make_first  
├── tslib  
│   ├── main.cpp  
│   ├── tslib.json  
│   └── tslib.pro  
└── tuiotouch  
    ├── main.cpp  
    ├── Makefile  
    ├── qoscbundle.cpp  
    ├── qoscbundle_p.h  
    ├── qoscmessage.cpp  
    ├── qoscmessage_p.h  
    ├── qtuiocursor_p.h  
    ├── qtuiohandler.cpp  
    ├── qtuiohandler_p.h  
    ├── qtuio_p.h  
    ├── README.md  
    ├── tuiotouch.json  
    └── tuiotouch.pro

bitbake 默认不编译 tslib ，编译启用的 plugins 有

➜  git git:(c619d2d) ✗ pwd  
/home/samba/sda3/freescale/3.14.52/yocto-L3.14.52/fb/tmp/work/cortexa9hf-vfp-neon-mx6qdl-poky-linux-gnueabi/qtbase/5.5.0+gitAUTOINC+c619d2daac-r0/git  
➜  git git:(c619d2d) ✗ tree plugins 
plugins  
├── bearer  
│   ├── libqconnmanbearer.so  
│   ├── libqgenericbearer.so  
│   └── libqnmbearer.so  
├── egldeviceintegrations  
│   └── libqeglfs-viv-integration.so  
├── generic  
│   ├── libqevdevkeyboardplugin.so  
│   ├── libqevdevmouseplugin.so  
│   ├── libqevdevtabletplugin.so  
│   ├── libqevdevtouchplugin.so  
│   └── libqtuiotouchplugin.so  
├── imageformats  
│   ├── libqgif.so  
│   ├── libqico.so  
│   └── libqjpeg.so  
├── platforminputcontexts  
│   └── libibusplatforminputcontextplugin.so  
└── platforms  
    ├── libqeglfs.so  
    ├── libqminimalegl.so  
    ├── libqminimal.so  
    └── libqoffscreen.so

bitbake的fsl-image-qt5依赖于qtbase，qtbase的recipe文件位于 /home/samba/sda3/freescale/3.14.52/yocto-L3.14.52/sources/meta-qt5/recipes-qt/qt5/qtbase_git.bb

$ cat /home/samba/sda3/freescale/3.14.52/yocto-L3.14.52/sources/meta-qt5/recipes-qt/qt5/qtbase_git.bb

  
......  


  

PACKAGECONFIG_RELEASE ?= "release"  
# This is in qt5.inc, because qtwebkit-examples are using it to enable ca-certificates dependency  
# PACKAGECONFIG_OPENSSL ?= "openssl"  
PACKAGECONFIG_DEFAULT ?= "dbus udev evdev widgets tools libs"  

PACKAGECONFIG ?= " \  
    ${PACKAGECONFIG_RELEASE} \  
    ${PACKAGECONFIG_DEFAULT} \  
    ${PACKAGECONFIG_OPENSSL} \  
    ${PACKAGECONFIG_GL} \  
    ${PACKAGECONFIG_FB} \  
    ${PACKAGECONFIG_X11} \  
    ${PACKAGECONFIG_FONTS} \  
    ${PACKAGECONFIG_SYSTEM} \  
    ${PACKAGECONFIG_MULTIMEDIA} \  
    ${PACKAGECONFIG_DISTRO} \  
"  

PACKAGECONFIG[release] = "-release,-debug"  
PACKAGECONFIG[developer] = "-developer-build"  
PACKAGECONFIG[sm] = "-sm,-no-sm"  
PACKAGECONFIG[tests] = "-make tests,-nomake tests"  
PACKAGECONFIG[examples] = "-make examples -compile-examples,-nomake examples"  
PACKAGECONFIG[tools] = "-make tools,-nomake tools"  
# only for completeness, configure will add libs even if you try to explicitly remove it  
PACKAGECONFIG[libs] = "-make libs,-nomake libs"  
# accessibility is required to compile qtquickcontrols  
PACKAGECONFIG[accessibility] = "-accessibility,-no-accessibility"  
PACKAGECONFIG[glib] = "-glib,-no-glib,glib-2.0"  
# use either system freetype or bundled freetype, if you disable freetype completely  
# fontdatabases/basic/qbasicfontdatabase.cpp will fail to build and system freetype  
# works only together with fontconfig  
PACKAGECONFIG[freetype] = "-system-freetype,-freetype,freetype"  
PACKAGECONFIG[harfbuzz] = "-system-harfbuzz,-no-harfbuzz,harfbuzz"  
PACKAGECONFIG[jpeg] = "-system-libjpeg,-no-libjpeg,jpeg"  
PACKAGECONFIG[libpng] = "-system-libpng,-no-libpng,libpng"  
PACKAGECONFIG[zlib] = "-system-zlib,-qt-zlib,zlib"  
PACKAGECONFIG[pcre] = "-system-pcre,-qt-pcre,pcre"  
PACKAGECONFIG[gl] = "-opengl desktop -no-eglfs,,virtual/libgl"  
PACKAGECONFIG[gles2] = "-opengl es2 -eglfs,,virtual/libgles2 virtual/egl"  
PACKAGECONFIG[tslib] = "-tslib,-no-tslib,tslib"  
PACKAGECONFIG[cups] = "-cups,-no-cups,cups"  
PACKAGECONFIG[dbus] = "-dbus,-no-dbus,dbus"  
PACKAGECONFIG[xcb] = "-xcb -xcb-xlib -system-xcb,-no-xcb,libxcb xcb-util-wm xcb-util-image xcb-util-keysyms xcb-util-renderutil"  
PACKAGECONFIG[sql-ibase] = "-sql-ibase,-no-sql-ibase"  
PACKAGECONFIG[sql-mysql] = "-sql-mysql,-no-sql-mysql,mysql5"  
PACKAGECONFIG[sql-psql] = "-sql-psql,-no-sql-psql,postgresql"  
PACKAGECONFIG[sql-odbc] = "-sql-odbc,-no-sql-odbc"  
PACKAGECONFIG[sql-oci] = "-sql-oci,-no-sql-oci"  
PACKAGECONFIG[sql-tds] = "-sql-tds,-no-sql-tds"  
PACKAGECONFIG[sql-db2] = "-sql-db2,-no-sql-db2"  
PACKAGECONFIG[sql-sqlite2] = "-sql-sqlite2,-no-sql-sqlite2,sqlite"  
PACKAGECONFIG[sql-sqlite] = "-sql-sqlite -system-sqlite,-no-sql-sqlite,sqlite3"  
PACKAGECONFIG[xcursor] = "-xcursor,-no-xcursor,libxcursor"  
PACKAGECONFIG[xinerama] = "-xinerama,-no-xinerama,libxinerama"  
PACKAGECONFIG[xinput] = "-xinput,-no-xinput"  
PACKAGECONFIG[xinput2] = "-xinput2,-no-xinput2,libxi"  
PACKAGECONFIG[xfixes] = "-xfixes,-no-xfixes,libxfixes"  
PACKAGECONFIG[xrandr] = "-xrandr,-no-xrandr,libxrandr"  
PACKAGECONFIG[xrender] = "-xrender,-no-xrender,libxrender"  
PACKAGECONFIG[xshape] = "-xshape,-no-xshape"  
PACKAGECONFIG[xsync] = "-xsync,-no-xsync"  
PACKAGECONFIG[xvideo] = "-xvideo,-no-xvideo"  
PACKAGECONFIG[openvg] = "-openvg,-no-openvg"  
PACKAGECONFIG[iconv] = "-iconv,-no-iconv,virtual/libiconv"  
PACKAGECONFIG[xkb] = "-xkb,-no-xkb -no-xkbcommon,libxkbcommon"  
PACKAGECONFIG[evdev] = "-evdev,-no-evdev"  
PACKAGECONFIG[mtdev] = "-mtdev,-no-mtdev,mtdev"  
# depends on glib  
PACKAGECONFIG[fontconfig] = "-fontconfig,-no-fontconfig,fontconfig"  
PACKAGECONFIG[gtkstyle] = "-gtkstyle,-no-gtkstyle,gtk+"  
PACKAGECONFIG[directfb] = "-directfb,-no-directfb,directfb"  
PACKAGECONFIG[linuxfb] = "-linuxfb,-no-linuxfb"  
PACKAGECONFIG[mitshm] = "-mitshm,-no-mitshm,mitshm"  
PACKAGECONFIG[kms] = "-kms,-no-kms,virtual/mesa virtual/egl"  
# needed for qtwebkit  
PACKAGECONFIG[icu] = "-icu,-no-icu,icu"  
PACKAGECONFIG[udev] = "-libudev,-no-libudev,udev"  
# use -openssl-linked here to ensure that RDEPENDS for libcrypto and libssl are detected  
PACKAGECONFIG[openssl] = "-openssl-linked,-no-openssl,openssl"  
PACKAGECONFIG[alsa] = "-alsa,-no-alsa,alsa-lib"  
PACKAGECONFIG[pulseaudio] = "-pulseaudio,-no-pulseaudio,pulseaudio"  
PACKAGECONFIG[nis] = "-nis,-no-nis"  
PACKAGECONFIG[widgets] = "-widgets,-no-widgets"  
PACKAGECONFIG[libproxy] = "-libproxy,-no-libproxy,libproxy"  
PACKAGECONFIG[libinput] = "-libinput,-no-libinput,libinput"  

QT_CONFIG_FLAGS += " \  
    -shared \  
    -silent \  
    -no-pch \  
    -no-rpath \  
    -pkg-config \  
    ${EXTRA_OECONF} \  
"  

......  

# qtbase is exception, configure script is using our get(X)QEvalMakeConf and setBootstrapEvalVariable functions to read it from shell  
export OE_QMAKE_COMPILER  
export OE_QMAKE_CC  
export OE_QMAKE_CFLAGS  
export OE_QMAKE_CXX  
export OE_QMAKE_CXXFLAGS  
export OE_QMAKE_LINK  
export OE_QMAKE_LDFLAGS  
export OE_QMAKE_AR  
export OE_QMAKE_STRIP  

do_configure() {  
    # we need symlink in path relative to source, because  
    # EffectivePaths:Prefix is relative to qmake location  
    if [ ! -e ${B}/bin/qmake ]; then  
        mkdir -p ${B}/bin  
        ln -sf ${OE_QMAKE_QMAKE_ORIG} ${B}/bin/qmake  
    fi  

    ${S}/configure -v \  
        -opensource -confirm-license \  
        -sysroot ${STAGING_DIR_TARGET} \  
        -no-gcc-sysroot \  
        -prefix ${OE_QMAKE_PATH_PREFIX} \  
        -bindir ${OE_QMAKE_PATH_BINS} \  
        -libdir ${OE_QMAKE_PATH_LIBS} \  
        -datadir ${OE_QMAKE_PATH_DATA} \  
        -sysconfdir ${OE_QMAKE_PATH_SETTINGS} \  
        -docdir ${OE_QMAKE_PATH_DOCS} \  
        -headerdir ${OE_QMAKE_PATH_HEADERS} \  
        -archdatadir ${OE_QMAKE_PATH_ARCHDATA} \  
        -libexecdir ${OE_QMAKE_PATH_LIBEXECS} \  
        -plugindir ${OE_QMAKE_PATH_PLUGINS} \  
        -importdir ${OE_QMAKE_PATH_IMPORTS} \  
        -qmldir ${OE_QMAKE_PATH_QML} \  
        -translationdir ${OE_QMAKE_PATH_TRANSLATIONS} \  
        -testsdir ${OE_QMAKE_PATH_TESTS} \  
        -examplesdir ${OE_QMAKE_PATH_EXAMPLES} \  
        -hostbindir ${OE_QMAKE_PATH_HOST_BINS} \  
        -external-hostbindir ${OE_QMAKE_PATH_EXTERNAL_HOST_BINS} \  
        -hostdatadir ${OE_QMAKE_PATH_HOST_DATA} \  
        -platform ${OE_QMAKESPEC} \  
        -xplatform linux-oe-g++ \ 
        ${QT_CONFIG_FLAGS}  
  
    qmake5_base_do_configure  
}  
...

运行qt程序时可用-plugin参数显式指定加载的plugin，如./bin -plugin evdevtouch:dev/input/event0，也可使用环境变量控制
```
export QT_QPA_EGLFS_TSLIB=1 # 未测  
export QT_QPA_EGLFS_DISABLE_INPUT="1"  #禁用  
export QT_QPA_EVDEV_TOUCHSCREEN_PARAMETERS="/dev/input/event0:rotate=0"  
export QT_QPA_GENERIC_PLUGINS="evdevtouch:/dev/input/event0"
```
Input on eglfs and linuxfb without libinput
Parameters like the device node name can be set in the environment variables QT_QPA_EVDEV_MOUSE_PARAMETERS, QT_QPA_EVDEV_KEYBOARD_PARAMETERS and QT_QPA_EVDEV_TOUCHSCREEN_PARAMETERS. Separate the entries with colons. These parameters act as an alternative to passing the settings in the -plugin command-line argument, and with some backends they are essential: eglfs and linuxfb use built-in input handlers so there is no separate -plugin argument in use.
Additionally, the built-in input handlers can be disabled by setting QT_QPA_EGLFS_DISABLE_INPUT or QT_QPA_FB_DISABLE_INPUT to 1.
用-platform指定qt程序运行的后端平台，一般X协议下的_platform_为xcb： The X protocol C-language Binding ，在tty下，可用linuxfb平台使用_framebuffer_来渲染gui，嵌入式系统上使用eglfs，除此之外，wayland后端也已经开始可用
- http://blog.qt.io/blog/2015/01/20/qt-weekly-23-qt-5-5-enhancements-for-linux-graphics-and-input-stacks/
- http://doc.qt.io/qt-5/embedded-linux.html
测试后发现板子上的qt程序不会产生ToucEnd事件，而在desktop机子上指定QT_QPA_PLATFORM=linuxfb 和-plugin evdevtouch:/dev/input/eventX时却能正常获得TouchEnd消息，~~猜想问题有可能来自于触摸屏或linux内核或qt库本身~~,进一步测试发现在desktop上的触摸屏程序支持多点触控，怀疑问题来自于触摸屏驱动。
最后确认问题来自驱动，已解决

解决思路步骤记录

发现板上触摸屏行为异常，表现为：
1. 界面上会显示指针，一般触屏应用从来没见过有指针。
2. 指针移动幅度极大，触摸屏上轻微移动就会造成指针的极大偏移，大概两三厘米的位移指针就会横跨屏幕。
3. 在屏幕上轻点，指针所指位置的按钮或菜单却无响应。
怀疑指针偏移是未校准所致，一般传统校准使用 tslib tools 中的 ts_calibrate进行，于是尝试编译移植 tslib 库。
tslib移植成功后运行校准和测试程序均正常，然而在运行qt程序的时候情况依旧。
找到qt库下的demo源码，将其编译后放到板上作为测试。
尝试摸清qt与tslib的关系，在翻阅源码目录的过程中发现yocto-L3.14.52/fb/tmp/work/cortexa9hf-vfp-neon-mx6qdl-poky-linux-gnueabi/qtbase/5.5.0+gitAUTOINC+c619d2daac-r0/git/src/plugins/generic/evdevtouch/README，该readme提到可用 -plugin 指定加载插件，同时从qt官网文档（上面链接）得知用环境变量控制插件加载
用-plugin evdevtouch加载evdevtouch后触摸屏demo可以工作，但同时鼠标指针也会存在，测试排查后发现鼠标指针由evdevmouse呈现，与evdevtouch相互独立。
在只启用evdevtouch时，触摸屏demo工作仍然不正常，表现为：
1. 位置和精度正常
2. fingerpaint demo，每次触摸会更换画笔颜色，然而程序测试无此表现
3. 修改源码打出 log 观察TouchEvent的传递
4. 发现只有第一次触摸到的 widget 能正常接受TouchEvent，之后无论移除范围还是抬起重新在其它widget上点击仍只有第一个widget收到event
5. 用定时设置焦点等方式确认该现象的产生与控件焦点无关，不应该是 qt 本身的机制
6. 观察 log ，发现只产生 TouchBegin 和 TouchUpdate event ，多方google后完全没有人出现过**不产生TouchEnd事件**的情况
7. 同时观察到电容屏却不支持多点触控，这在3.x的linux内核上说不通，Android 2.2时代 Android 内核 刚迈入3.x时代，却无一例外很容易就支持多点触控，初步怀疑到驱动
尝试用-plugin tslib启动qt程序，提示该plugin不存在，尝试更改qt的recipe使之编译 tslib plugin
多方尝试无果，而用桌面电脑和树莓派作为参考系统，同一触摸板都表现正常，也都支持多点触控，再经查找文档得知tslib 历史较老，多为电阻屏所使用，qt不编译tslib大概出于放弃老旧特性，而不是照顾兼容性的考虑，放弃tslib，转而验证驱动有问题的思路
经指点，找到 linux-imx 下的内核源码，bitbake linux-imx -c menuconfig编辑内核配置发现其中关于 hid-multitouch 和 **device->input-> Touchscreens->***相关的绝大部分特性都没有编入，基本确定问题由驱动所致。
cat /proc/bus/input/devices后得知触摸屏生产制造公司为 ILITEK，将Device Drivers->Input->Touchscreens下的 Ilitek ILI210X based touchscreen 和 USB Touchscreen Driver作为built-in特性
重新编译内核并用新内核启动，再测试demo程序，表现与桌面机和树莓派上一致正常，至此问题解决