How to automate chrome with playwright

First open Chrome like this:

C:\Program Files\Google\Chrome\Application\chrome.exe 

--remote-debugging-port=9222

--user-data-dir="C:\Users\User\AppData\Local\Google\Chrome\User Data\Profile 2"

--save-storage

Find the profile dir with  chrome://version/

Then manually log in to the website you want to automate.

Now run:  npx playwright codegen --target=javascript -o script.js https://example.com

...and it will record your actions.

After that, Playwright can connect to chrome and repeat the actions.

More notes

• If you need exact mouse trajectories or raw timestamps: Use rrweb to record events in-page and replay to produce precise input traces. Or capture DevTools Protocol Input events (use a small recorder that listens to Input.dispatchMouseEvent and Input.dispatchKeyEvent) and convert to Playwright calls (page.mouse.move + page.mouse.down/up + page.keyboard.type).

Automating mouse movement and clicks on Linux

Windows has AutoHotkey (AHK), and Linux has xdotool.

But xdotool 3.20160805.1 doesn't work on Wayland at all. It silently fails. Turns out xdotool only works on X11, which is no longer the default for Ubuntu.

In 2024, using Ubuntu 22, this version of xdotool is the latest one available by running `apt install xdotool`.

To find out if you're using Wayland:

$ loginctl show-session 2 -p Type
Type=wayland
$ echo $XDG_SESSION_TYPE
wayland

To switch to X11, edit/etc/gdm3/custom.conf, uncomment the line WaylandEnable=false, and reboot.

Now xdotool will work perfectly.

xdotool README does mention some alternatives:

• ydotool (available in apt) -- last updated January 2023, also didn't work for me on Wayland or X11
• dotool -- updated more recently, but not packaged for Ubuntu, and building from source requires Go, which is quite heavy so it's easier to just switch to X11 for xdotool.