14 August 2026

A Farmer That Never Sleeps: Automating Hay Day With ADB and OpenCV

A look under the hood of my Hay Day bot: why I ditched input swipe for raw sendevent events, and the dirty tricks that make the sowing gesture smooth.

I wrote a bot that sows a field in Hay Day without me lifting a finger. No mouse hijacking, no pixel-poking on the Windows desktop: everything runs purely through ADB into a MEmu emulator, and the field is found with OpenCV. Sounds simple. It wasn't.

This isn't a tutorial with a happy path. It's the story of the three places where the obvious solution broke, and the workarounds that finally made it smooth.

The bot recipe in short

The pipeline is straightforward:

Grab a screenshot via ADB (screencap).
Detect the soil by colour with OpenCV.
Tap the centre so the camera focuses.
Pick up the wheat tool (long-press) and drag it in a zigzag across the field.

Step 4 is where it gets interesting. Sowing a field isn't one tap, it's a single long, unbroken drag that passes over every bit of dirt. And that's where I hit the first wall.

Workaround 1: dumping `input swipe` for raw evdev events

The standard way to send a swipe on Android is adb shell input swipe. Fine for a single swipe. But a sowing path is made of dozens of segments, and every input swipe call is a complete gesture of its own: pen down, move, pen up. Chain them together and the finger lifts between each segment and re-anchors the touch pointer. The result: stuttering, broken swipes and a half-sown field.

The fix is to dive one layer deeper, into the Linux evdev layer where Android gets its touch input. The sendevent command writes raw input events straight to /dev/input/eventN. By sending DOWN once, then a stream of MOVE events, and only UP at the very end, you get one truly unbroken gesture.

The protocol is Type B (slot-based multitouch). Here's what a touch-DOWN looks like as individual events:

# ── touch DOWN ───────────────────────────────────────────────────────
hx0, hy0 = self._to_hw(*android_points[0])
lines += [
    self._se(EV_ABS, ABS_MT_SLOT,         0),   # slot 0 = first finger
    self._se(EV_ABS, ABS_MT_TRACKING_ID,  1),   # positive id = finger present
    self._se(EV_ABS, ABS_MT_POSITION_X,   hx0),
    self._se(EV_ABS, ABS_MT_POSITION_Y,   hy0),
]
if self.pressure:
    lines.append(self._se(EV_ABS, ABS_MT_PRESSURE, self.pressure))  # many devices require pressure > 0
lines += [
    self._se(EV_KEY, BTN_TOUCH,            1),
    self._se(EV_ABS, ABS_X,                hx0),  # single-touch compat layer
    self._se(EV_ABS, ABS_Y,                hy0),
    self._syn(),                                  # commit this frame
]

Each sendevent line is literally one EV_ABS/EV_KEY/EV_SYN triple. The _syn() at the end is crucial: without a SYN_REPORT, the kernel doesn't know the frame is complete and nothing happens.

A few traps I ran into along the way:

ABS_MT_PRESSURE often has to be > 0, otherwise the device doesn't register the touch.
Lifting the finger is done with an unsigned -1 as the tracking id, i.e. 4294967295. That's the evdev convention for "slot empty".
ABS_X/ABS_Y (the old single-touch compat events) are only needed at touch-down. Sending them on every MOVE doubles the number of sendevent calls for nothing. Dropping them halves the script time:

# Only ABS_MT_POSITION_X/Y + SYN_REPORT per move (3 calls).
# ABS_X/ABS_Y are compat events only needed at touch-down;
# dropping them from moves halves the sendevent call count.
for ax, ay in android_points[1:]:
    hx, hy = self._to_hw(ax, ay)
    if hx == prev_hx and hy == prev_hy:
        continue  # skip duplicates, saves more calls
    lines += [
        self._se(EV_ABS, ABS_MT_POSITION_X, hx),
        self._se(EV_ABS, ABS_MT_POSITION_Y, hy),
        self._syn(),
    ]
    if delay > 0:
        lines.append(f"sleep {delay:.4f}")
    prev_hx, prev_hy = hx, hy

Why a shell script, not individual ADB calls

Sending each sendevent separately over ADB would mean hundreds of round-trips: slow and with flaky timing. For a large path the script easily runs to hundreds of lines. Passing that whole string on the command line slams into ARG_MAX.

The workaround: write the full script to a file, push it to the device, and run it there with a single sh call. All timing happens on the device itself, exactly as intended.

# Push script to device (avoids ARG_MAX limit, minimises round-trips)
self.adb.run("-s", self.adb.device_address, "push", local_tmp, REMOTE_SCRIPT_PATH)
...
self.adb.shell("sh", REMOTE_SCRIPT_PATH, check=False, timeout=timeout)

I estimate the timeout up front: each sendevent costs about 6 ms because it spawns a child process on Android, plus all the sleep time, plus some headroom.

The long-press hack to pick up items

In Hay Day you pick up tools with a long press, not a tap. I simulate that by simply staying still after the touch-DOWN but before the first move:

# ── hold at start (long-press to pick up item) ────────────────────
lines.append(f"sleep {hold_duration:.3f}")   # ~0.8s so Hay Day registers the pick-up

No separate long-press call needed; it's just a sleep in the middle of the one big gesture.

Workaround 2: from a messy colour mask to a clean field

Finding the field happens by colour. I take two sample images of soil, compute their average colour, and build a mask of all pixels close to it. Two masks, because soil doesn't look the same everywhere:

diff1 = np.abs(screen - self.template1_color)
mask1 = (np.mean(diff1, axis=2) < self.color_threshold).astype(np.uint8) * 255

diff2 = np.abs(screen - self.template2_color)
mask2 = (np.mean(diff2, axis=2) < self.color_threshold).astype(np.uint8) * 255

current_mask = cv2.bitwise_or(mask1, mask2)   # looks like soil A OR soil B

The raw mask is full of holes: the furrows between crops break the image into separate horizontal stripes. A morphological close with a 15x15 kernel glues those stripes back into one solid region:

kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (15, 15))
current_mask = cv2.morphologyEx(current_mask, cv2.MORPH_CLOSE, kernel)

After that I filter contours by minimum area (5000 px) to throw out snow patches and noise, and take the largest one. The real trick is minAreaRect: it draws the smallest rotated rectangle around the contour, filling in missing corners into the true field shape.

The bug that marked half the screen as "field"

There was a nasty bug here. My first getSoilBounds computed the bounds from the full mask. Problem: one stray false-positive pixel on the far left and one on the far right, and suddenly the bounding box stretches across the entire screen width. The sowing path then tried to seed half the screen.

The workaround: don't compute bounds from the mask, compute them from the largest valid contour. Noise pixels then fall outside the contour and don't count.

def getSoilBounds(self, screen):
    """
    Uses the bounding rect of the largest valid contour, NOT the full mask.
    The full mask let stray false-positive pixels at the screen edges
    stretch min_x/max_x to the full screen width.
    """
    _, mask, center, _ = self.detect(screen)
    if mask is None or center is None:
        return None

    contours, _ = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
    valid = [c for c in contours if cv2.contourArea(c) > self.min_contour_area]
    if not valid:
        return None

    largest = max(valid, key=cv2.contourArea)
    x, y, w, h = cv2.boundingRect(largest)
    return (x, y, x + w, y + h, center[0], center[1])

A tiny detail, but the difference between a neatly sown field and a bot smearing seeds randomly across the screen.

Workaround 3: a zigzag without sharp corners

With the field mapped out, I need a path that touches every spot without lifting the finger. The naive approach, left-right-left with sharp corners at the edge, stutters: at a 90-degree turn the game sometimes loses the gesture.

My solution is a raster of horizontal rows, but at each edge the finger makes a tight semicircle into the next row. No sharp corner, no wasted movement, and the turns bulge just outside the field (the game ignores touches outside the soil, so that's fine):

def right_uturn(x_edge, y_top):
    """
    Semicircle at the RIGHT edge:
      (x_edge, y_top) -> arc right -> (x_edge, y_top + 2r)
    Center = (x_edge, y_top + r), radius = r, clockwise.
    """
    cy_t = y_top + r
    n    = max(8, int(math.pi * r / point_spacing))
    return [(int(x_edge + r * math.cos(-math.pi/2 + math.pi*i/n)),
             int(cy_t   + r * math.sin(-math.pi/2 + math.pi*i/n)))
            for i in range(n + 1)]

The U-turn radius is exactly half the row spacing (r = spacing // 2). That makes each row overlap the previous turn by exactly the turn radius, so no strip is ever skipped. The rows themselves I clamp inside the field bounds; only the U-turns are allowed to poke outside:

going_right = True
y = min_y
while y <= max_y:
    if going_right:
        pts += row(min_x, max_x, y)
        next_y = y + spacing
        if next_y <= max_y:
            pts += right_uturn(max_x, y)   # connects row y to row y+spacing
    else:
        pts += row(max_x, min_x, y)
        next_y = y + spacing
        if next_y <= max_y:
            pts += left_uturn(min_x, y)
    if next_y > max_y:
        break
    y = next_y
    going_right = not going_right

Bonus: the window that never freezes

A debug viewer that hangs is useless. So all the ADB work (screenshots, taps, the slow drag) runs in a separate thread; the main thread does nothing but the cv2 window loop and stays responsive at all times.

"""
Bot thread  — all ADB work (screenshots, tap, drag)
Main thread — cv2 window loop, never blocks, always responsive
"""
...
t = threading.Thread(target=bot_main, args=(view,), daemon=True)
t.start()
while t.is_alive():
    view.tick()   # main thread pumps the window

cv2's GUI loop has to live on the main thread, so that choice wasn't optional: it's the only layout that gives both a live overlay and a non-blocking bot.

What I take away from it

The common thread: the obvious API is often too coarse-grained. input swipe can't do an unbroken gesture. A colour mask can't delineate a field. A sharp corner can confuse the game. Every time, the solution was to go one layer deeper (raw evdev events, contour bounds, geometric turns) and sand down the rough edge with a targeted workaround.

It's still a prototype: harvesting isn't in there yet, and the detector is sensitive to how the soil looks. But it sows, and it sows smoothly. And the most interesting part isn't the bot itself, it's what you run into underneath the moment you have to go one layer deeper.