Pet Sim 99 Autofish

1. The start

   Hours of watching the water

   Pet Simulator 99, the Roblox game by BIG Games, is built around collecting rare pets, and it asks for a lot of grinding to get them. Its fishing minigame is a good example: the best catches come from whirlpools that appear and drift across the water, and chasing them means hours of staring at the screen, waiting.

2. The goal

   Let the computer do the watching

   I wanted to see how little code it takes to watch the water for me: spot a whirlpool, put the cursor on it, follow it as it drifts, and keep going for as long as the game is open. A proof of concept for screen-reading automation, and an excuse to get properly into computer vision.

3. The adventure

   From pixels to a precise action

   I wrote a Python loop that captures the fishing area, compares it with reference images of whirlpools using OpenCV's template matching, and moves the cursor onto the best match, over and over. An external autoclicker takes care of casting, which also keeps the session active so the game doesn't disconnect.

   • 
   • 
   • 
   • 
   • 
   • 
   • 
   • 
   • 
   • 

4. The difficulties

   Moving targets in moving water

   Whirlpools drift, and the water around them is animated, which is exactly the kind of scene that fools a naive matcher. The tool needed several reference images and extra checks on shape and colour to reject false hits, and it had to keep up in real time without eating the CPU the game needs.

5. The end

   An overnight run, and a few Huge pets

   It ran unattended overnight without interruption and landed several Huge pets, among the rarest in the game. It's a minimal loop that reads the screen and turns a visual cue into a precise action: the same pattern that sits behind the rest of my automation work.

• Image recognition

  Computer vision spots whirlpools, the best fishing spots, as soon as they appear on the water.

• Precise cursor control

  Puts the cursor right on the detected whirlpool, where the loot is best.

• Hands-off operation

  Watches and adjusts continuously, with no one at the keyboard.

• Works with an autoclicker

  Pairs with an external autoclicker for casting, which also keeps the game from disconnecting.

• Better loot

  Aiming at whirlpools targets the best rewards, including rare pets.

• Real time

  Analyses the screen continuously and reacts immediately when the water changes.

The game behind it

Pet Simulator 99 is a popular Roblox game by BIG Games about collecting pets of different rarities, trading them and joining clans. Its constant grind is what inspired this tool.

The loop

1. Capture

   Grab the fishing area of the game window.

2. Detect

   Look for whirlpools with template matching, plus checks on shape and colour.

3. Locate

   Work out the exact coordinates of the best match.

4. Aim

   Move the cursor onto it.

5. Repeat

   Start over, so the cursor follows the whirlpools as they drift.

Recognising a whirlpool

The matcher compares the screen against these reference images. Several of them are needed because a whirlpool looks different depending on its size, its position and the water around it.

• Template matching

  OpenCV

  Compares regions of the screen with the predefined whirlpool references.

• Contour detection

  Shape

  Looks for the circular patterns a whirlpool draws on the water.

• Colour analysis

  Colour

  Checks for the colours that go with high-value fishing spots, to reject false hits.

• Motion tracking

  Movement

  Follows dynamic elements like drifting whirlpools and animated water.

The stack

• OpenCV

  Computer vision

  Template matching and the image processing around it.

• Pillow

  Images

  Image manipulation and analysis of the captures.

• PyAutoGUI

  Screen and cursor

  Screen capture and precise cursor positioning.

• NumPy

  Arrays

  The numerical work on image data, fast enough for a real-time loop.

Skills I gained

• Computer vision

  Image recognition

  Template matching, contours and colour analysis on a noisy, animated scene.

• Real-time screen analysis

  Automation

  A capture-and-react loop that keeps up without slowing the game down.

• Coordinate mapping

  Cursor control

  Turning a match in an image into an exact position on screen.

• Library integration

  Python

  OpenCV, Pillow, NumPy and PyAutoGUI working together in one small tool.