最新消息:20210917 已从crifan.com换到crifan.org

【未解决】用OpenCV检测游戏界面中元素并用uiautomator2去点击按钮

按钮 crifan 713浏览 0评论
折腾:
【未解决】uiautomator2中如何获取到欢乐大作战游戏界面中的元素
期间,去试试OpenCV
去检测界面中的元素
比如 主线关卡 背包 英雄 任务 等等
opencv python
opencv-python · PyPI
OpenCV-Python Tutorials — OpenCV-Python Tutorials 1 documentation
Introduction to OpenCV-Python Tutorials — OpenCV-Python Tutorials 1 documentation
OpenCV本身是C++实现的
此处提供Python(的打包wrapper的)接口
还支持了Numpy,其提供了类似Matlab的语法
所有的OpenCV数组结构,都可以和Numpy的数组互相转换。
-》OpenCV中的数组支持Numpy的数组的所有操作。
而且还能用其他支持Numpy的库,比如SciPy,Matplotlib
所以OpenCV-Python很适合对于计算机视觉方面的问题的快速原型验证
用OpenCV-Python之前,最好对于Python,尤其是Numpy很熟悉,这样才能写出性能好的代码。
Gui Features in OpenCV — OpenCV-Python Tutorials 1 documentation
Getting Started with Images — OpenCV-Python Tutorials 1 documentation
先去安装OpenVC
但是并没有找到如何在mac中安装
OpenCV: Introduction to OpenCV-Python Tutorials
OpenCV: Introduction
OpenCV Python教程(1、图像的载入、显示和保存) – sunny2038的专栏 – CSDN博客
给深度学习入门者的Python快速教程 – 番外篇之Python-OpenCV – 知乎
“- object:目标检测模块,包含级联分类和Latent SVM”
pip install opencv-python
那直接去
➜  autoTestAndroidGameHappyBigBattle pipenv install opencv-python
Courtesy Notice: Pipenv found itself running within a virtual environment, so it will automatically use that environment, instead of creating its own for any project. You can set PIPENV_IGNORE_VIRTUALENVS=1 to force pipenv to ignore that environment and create its own instead. You can set PIPENV_VERBOSITY=-1 to suppress this warning.
Installing opencv-python…
Adding opencv-python to Pipfile's [packages]…
✔ Installation Succeeded 
Pipfile.lock (bfc05b) out of date, updating to (ae4f88)…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
✔ Success! 
Updated Pipfile.lock (bfc05b)!
Installing dependencies from Pipfile.lock (bfc05b)…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 27/27 — 00:00:06
看看依赖:
➜  autoTestAndroidGameHappyBigBattle pipenv graph
opencv-python==4.1.2.30
  - numpy [required: >=1.14.5, installed: 1.17.4]
然后去导入代码试试
➜  autoTestAndroidGameHappyBigBattle python
Python 3.7.3 (default, May 22 2019, 10:55:14) 
[Clang 10.0.1 (clang-1001.0.46.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import cv2
>>> print(cv2.__version__)
4.1.2
安装好了。
此处需要先去uiautomator2中截图
参考
https://zhuanlan.zhihu.com/p/84523953
screen = self.d.screenshot(format="opencv")
再去考虑如何从截图中识别想要识别的部分
OpenCV: OpenCV modules
curScreen = d.screenshot(format="opencv")
logging.info("curScreen=%s", curScreen)
结果:
看起来是可以获取到了截屏图片
那接着去尝试用cv2打开看看截图图片
拷贝出:
array([[[145, 111,  51],
        [144, 110,  50],
        [143, 109,  49],
        ...,
        [155, 118,  62],
        [154, 117,  61],
        [154, 117,  61]],


       [[144, 110,  50],
        [144, 110,  50],
        [143, 109,  49],
        ...,
        [155, 118,  62],
        [154, 117,  61],
        [154, 117,  61]],


       [[143, 109,  49],
        [143, 109,  49],
        [144, 110,  51],
        ...,
        [155, 118,  62],
        [154, 117,  61],
        [154, 117,  61]],


       ...,


       [[177, 119,  47],
        [178, 120,  48],
        [178, 120,  48],
        ...,
        [255, 255, 255],
        [255, 255, 255],
        [255, 255, 255]],


       [[176, 118,  46],
        [177, 119,  47],
        [177, 119,  47],
        ...,
        [255, 255, 255],
        [255, 255, 255],
        [255, 255, 255]],


       [[176, 118,  46],
        [177, 119,  47],
        [177, 119,  47],
        ...,
        [255, 255, 255],
        [255, 255, 255],
        [255, 255, 255]]], dtype=uint8)

转载请注明:在路上 » 【未解决】用OpenCV检测游戏界面中元素并用uiautomator2去点击按钮

发表我的评论
取消评论

表情

Hi,您需要填写昵称和邮箱!

  • 昵称 (必填)
  • 邮箱 (必填)
  • 网址
90 queries in 0.183 seconds, using 23.36MB memory