Need help with touch screen macro

Citra Support
#1

Issue:
Hello, I’m trying to do a script to click on a part of the touch screen to avoid switching between gamepad and mouse in Zelda OOT.
My script use image recongnition to find the right button and I will asign a pad button do do the script.
The problem is that Citra does not seems to support pyautogui. Do you have I idea how I can do that? Here is ma Python code for testing.

import pyautogui
import time

time.sleep(3)

Recherche puis clic bouton Item

itemLoc = pyautogui.center(pyautogui.locateOnScreen (‘item.png’, confidence=0.5))
pyautogui.click(itemLoc.x, itemLoc.y)

System Information

  • Operating System: Windows 10
  • CPU: AMD FX 8350
  • GPU: Geforce 660ti
  • Citra Version (found in title bar): Nightly 1203
  • Game: Zelda OOT 3D

Diagnostic Log
Upload your log file as an attachment by dragging & dropping.

In order to save a copy of the log, follow this guide:
https://community.citra-emu.org/t/how-to-upload-the-log-file/296

#2

In order for us to provide better support, we need to see the log generated by Citra. This guide will walk you through how you can obtain the log file: How to Upload the Log File.

#3

just use joytokey and map the button with a mouse click + coordinates of the mouse, im using it so that when i click L2 or ZL it clicks the ocarina and R2 or ZR to click items, select to view mode, and R analog to click I and II buttons

#4

Hoo I don’t knew Joy2Key could do that, wil give it a try. Can it do image recognition too? Because I’m playing on different layout (portrait via moonlight when not a home and landscape when at home)

#5

so youre playing on mobile

#6

I’m playing on PC through my mobile with a ps3 pad :slight_smile:

#7

Just tried, it does not works. The click is made on the return to desktop button. I think the fact that I use Nvidia Gamestream mess something with resolution and mouse coordinates

#8

that’s not a very practical idea, there are better ways to do that, either by finding the addresses of those buttons and call them or by sending a click event to the buttons position on screen.

#9

I have no idea how to do that :frowning: I wanted to made an image recognition to be able to change screen configuration, but even without that, just click at a fixed position on screen, it does not work.

#10

Hello. I’ve done a few tests and it’s not working with joy2key, it does not click the right place. Maybe because double screen or maybe because of streaming.
Anyway it’s not a solution for me because sometimes I stream 1080p and sometimes 360 or 480 (network issues in train and tramway), so the mouse coordinates changes.
It would be a good feature for citra to assign tactile screen zone to key :slight_smile:

#11

It may be easier by using AHK, for instance this simply code (source) works just fine on citra:

x := ;place your desired coordinates in these two variables.
y := 

f1::
 {
   mousegetpos, start_x, start_y
   mouseclick, left, %x%, %y%, 1, 0
   mousemove, %start_x%, %start_y%, 0
 }
return

and you can get the windows x,y,w,h with this function:
https://autohotkey.com/docs/commands/WinGetPos.htm#Example

now you can know the windows coordinates before sending the click event and properly calculate the desirable click position (i guess).

#12

Hello,

That’s what I have done just have two scripts one for 720p and one for 1080p.
When streaming at 360 or 480, the real resolution is 720p