Teaching AI to Read the Game: From Final Whistle to Faster Decisions
What a field hockey video tagging experiment revealed about prompt design, human validation and the future of customizable performance analysis.
By: Suman Roy
Every match creates valuable data, but in many sports organisations that data becomes usable only after hours of manual work. By the time the footage has been reviewed, tagged, clipped and reported, the best window for discussion and decision-making may already be closing.
This is not only an analysis challenge. It is an operational bottleneck that affects turnaround time, consistency and the ability to scale video tagging and sports interpretation across more teams, matches and competitions.
The hidden workload after the final whistle
When the match ends, the analyst’s work often continues. Important actions must be located in the footage, timestamps corrected, outcomes attached, clips created and reports prepared. The process demands specialist knowledge and sustained attention, but much of the first pass is repetitive. As match volume grows, organisations face a difficult choice: accept slower delivery, increase manual capacity or analyse fewer games.
The cost is not limited to analyst hours. Coaches and other decision-makers may have to wait for information that is already present in the video but has not yet been converted into a usable form. Valuable footage remains difficult to search, compare or reuse until someone completes the manual tagging workflow.
Testing AI-assisted tagging in SPAN
SPAN is our AI-powered sports analytics and performance analysis software, designed to accelerate post-match analysis. We used its AI-assisted video tagging workflow to explore whether a model could handle part of the repetitive first pass while keeping human expertise at the centre of the process.
Within SPAN, users can select the Google model used for an AI-assisted tagging run. The available options include Gemini 3.1 Pro Preview, Gemini 3.5 Flash, Gemini 3.5 Flash lite and Gemini 3.6 Flash. The hockey experiment documented in this article used Gemini 3.5 Flash for the recorded prompt runs.
Two additional parameters help define how the video is processed. Video Length allows the user to select the duration of the match that should be AI tagged, so the run can focus on a chosen section instead of processing the complete video. Video Quality tells the AI the quality of the uploaded footage, with the available settings Low, Medium, High and Very High. In the documented hockey tests, Video Quality was set to Medium.
The experiment used field hockey footage and a structured tagging panel covering circle entries, penalty corners, long corners, shot attempts, turnovers, field areas and match quarters. It was a practical test of how AI driven tech could identify events in context, rather than simply describe the video. The goal was to generate structured events that could be reviewed, filtered and connected to the underlying footage.
The first lesson: rules alone are not enough
The initial prompt introduced the teams, their jersey colours and attacking directions, then defined the hockey events the model was expected to identify. The result was limited: the model created only one Circle Area Entry tag, and its timestamp did not match the event. It did, however, identify the team and match quarter correctly.
That outcome exposed a gap between knowing the definition of an event and recognising it in video, a core challenge in sports interpretation. A human analyst automatically combines the rules with field markings, player movement, team shape, scoreboard information and the sequence of play. The model needed those observations to be translated into explicit instructions.
A prompt is not a one-time instruction.
Prompting as a continuous feedback loop
The goal was not to write one increasingly long prompt and hope the model eventually understood the game. A reliable prompt developed through iteration: review the output, identify the misunderstanding, convert that misunderstanding into a clearer instruction and test again.
The later prompts began to teach the AI how an experienced analyst watches hockey. We added visual information about the turf, field markings, attacking direction, team shape, player movement and referee signals. We also explained when an event should begin and end, which outcome should be attached and which similar-looking actions should be excluded.
Three mistakes that made the prompt better
Drawing the correct boundary.
The AI initially treated entry into the dotted semicircle as a genuine circle entry. The next prompt explicitly stated that only the solid white semicircle directly in front of the goal should count, while the dotted semicircle had to be excluded. That distinction improved the model’s ability to identify genuine circle entries.
Capturing the complete event.
The first result placed the tag at the wrong moment. We therefore added temporal instructions: begin the tag five seconds before the action and end it ten seconds after the action was completed. With visual and timing context combined, the model identified the action, the team, the outcome and four separate events within a one-minute sequence.
Preserving the build-up to a turnover.
The AI could recognise that possession had changed, but some turnover tags began only after the opponent had already received the ball. The prompt was refined so the event started before the interception or misplaced pass, retaining the build-up that caused the turnover rather than capturing only its final moment.
Separating similar restarts.
Some restarts near the backline were incorrectly classified as long corners. The prompt was expanded to describe the exact corner location and to exclude restarts taken from the outside line. This reduced false positives, although the distinction remained one of the harder edge cases.
Human validation remains essential
The model improved substantially, but it was not perfect. Difficult restart situations still caused confusion, and two actions were duplicated. These are not reasons to dismiss AI-assisted tagging; they are reasons to design the workflow correctly. AI-generated events should be treated as a fast first pass that is reviewed by someone with domain knowledge. Those reviewed events can also serve as high-quality data annotations for AI workflows because the corrections capture the practical judgement of experienced analysts.
The value is not in replacing the analyst. It is in reducing repetitive event identification so analysts can spend more time validating important moments, interpreting patterns and supporting decisions. Human expertise moves higher in the workflow, from collecting every data point manually to ensuring that the data is accurate and meaningful.
From AI output to Customizable Performance Analysis
With SPAN, teams can analyze sports video without leaving detections as isolated AI outputs. Once reviewed, they can become structured performance data, searchable video events and interactive, video-linked charts. A user can move from a number in a chart directly to the footage behind it, making the evidence easier to explore without waiting for every clip to be prepared manually.
MATCH VIDEO → AI-ASSISTED TAGGING → HUMAN VALIDATION → INTERACTIVE REPORTING → FASTER DECISIONS
For sports organisations, this changes more than turnaround time. It creates a foundation for customizable performance analysis: more footage can be analysed through a consistent process, match data can be reused across reviews, and less knowledge remains locked inside individual spreadsheets, laptops or one-off reports.
The wider lesson for sports AI
The experiment demonstrated that domain knowledge becomes useful to AI only when it is translated into observable instructions. Visual cues explain what the model should see. Temporal rules explain how the event unfolds. Context explains why the action matters. Exclusions explain which near-matches should not be tagged.
Prompt development is therefore not a setup task completed once at the beginning. It is part of the analysis workflow itself. Each error provides information about how the model is interpreting the sport and what the next instruction must clarify.
Conclusion
AI-assisted video tagging will not eliminate the need for expert review, and it should not be presented as fully automatic analysis. Its immediate opportunity is more practical: reduce the repetitive first pass, shorten the distance between footage and insight, and give specialists more time to work on interpretation rather than collection.
When domain knowledge is translated into clear visual, temporal and contextual instructions, AI can begin to identify complex sports actions with much greater precision. Combined with human validation and video-linked reporting in SPAN, those detections can become structured, reusable intelligence that helps sports organisations make faster and better-informed decisions.
SPAN AI-assisted tagging workflow: from match context and prompt design to AI-generated video events.
Appendix: The Four Prompts, Outcomes and Learnings
The prompt text below is reproduced exactly as it appeared in the experiment document, including the original wording, grammar and capitalisation. Outcomes and learnings have been edited only for clarity and readability.
Prompt 1
Prompt –
There are 2 teams playing a game of Field Hockey. One of them is called Team A, and the other is called Team B. Team A is wearing a blue jersey and is attacking from left to right on the screen, and Team B is wearing White Jersey attacking from right to left on the field.
The start of any event must align with the initiation of one of the 8 primary categories: Circle Area Entry Team A, Circle Area Entry Team B, Hockey Long Corner Team A, Hockey Long Corner Team B, Hockey Penalty Corner Team A, Hockey Penalty Corner Team B, Shot Attempt Team A, Shot Attempt Team B. Please ignore moments of no action, such as injury breaks, discussions with the referee, or when the ball is out of play. The end of an event occurs when the primary category no longer matches the current action or the phase of play shifts.
A Circle Area Entry happens when a player crosses the solid white semi-circle line; use the secondary category ‘Circle Area Entry Result’ to label the outcome as ‘Shot On Target’ (keeper saves the ball), ‘Goal’ (ball goes into the net), or ‘Shot Off Target’ (ball misses or is blocked by a non-goalkeeper). A Long Corner is awarded when the ball crosses the backline last touched by a defender (without intentional clearance); the referee signals this by raising one arm horizontally and pointing toward the corner flag. A Hockey Penalty Corner is awarded to the team in possession when a foul occurs inside the circle, an intentional foul happens in the 23-meter area, or the ball is intentionally hit over the backline; the referee signals this by pointing both arms horizontally toward the offending team’s goal. A Shot Attempt occurs when the attacking team shoots toward the goal, categorized as a Goal, Shot On Target, or Shot Off Target. Finally, use the secondary category ‘Hockey Quarters’ to record the current match quarter and the duration shown on the top-left of the screen.
Outcome
The AI generated only one Circle Area Entry tag. The timestamp did not match the actual event, but it correctly identified the team and the first match quarter.
Learning
Providing sporting definitions alone was not enough. The AI needed clearer visual references, contextual information and instructions about when each event should begin and end.
Prompt 2
Prompt –
Two teams are playing a game of Field Hockey on a turf which is completely blue in colour. One of them is called Team A, and the other is called Team B. Team A is wearing a blue jersey and is attacking from left to right on the screen, and Team B is wearing White Jersey attacking from right to left on the field.
The start of any event must align with the initiation of one of the 8 primary categories: Circle Area Entry Team A, Circle Area Entry Team B, Hockey Long Corner Team A, Hockey Long Corner Team B, Hockey Penalty Corner Team A, Hockey Penalty Corner Team B, Shot Attempt Team A, Shot Attempt Team B. Please ignore moments of no action, such as injury breaks, discussions with the referee, or when the ball is out of play. The end of an event occurs when the primary category no longer matches the current action or the phase of play shifts.
Circle entry is when a player of either the blue team or the white team enters the Semicircle area that appears in front of the goalpost by a solid white line. When there is a circle entry, you will see that the defending team will be very compact and in front of the goalpost. There is a secondary category, Circle Area Entry Result, which tells us about the next action that has happened after a player from either of the teams entered the area, which further has labels, which means the outcome of the Circle Area Entry, i.e., Shot On Target, Goal, Shot Target. A Shot On Target is when the player in possession of the ball, i.e., the attacking team, takes a shot towards the defending team’s goalpost and the defending team’s Goalkeeper saves it from going inside the goalpost. A Goal is when the Player from the attacking team takes a shot towards the opponent’s goalpost, and the opponent’s goalkeeper is not able to save the ball, and it goes inside the goalpost. A Shot Off Target is when the player of the attacking team takes a shot, but it doesn’t go near the defending team’s goalpost or is blocked by any of the opponent’s players other than the goalkeeper. When you see a Circle Area Entry tag at least 5 seconds before the entry as start time and 10 seconds after the action is completed as end time. A long corner is awarded when the ball goes over the defending team’s backline that is behind the goalpost line and was last touched by a defending player, provided it was not an intentional clearance, also the signal given by the referee that is on the field wearing a yellow colour jersey will signal by raising one arm horizontally and pointing with it directly toward the corner flag on the side of the field where the ball went out over the backline when you see a Long Corner tag at least 5 seconds before the pass is being made by the attacking team player from the corners as start time and 10 seconds after the action ends as end time. A Hockey penalty corner is awarded to the team in possession of the ball when the defending team player commits a foul inside the circle or makes an intentional foul inside the 23-meter area to identify the 23-meter area we should look at the solid white horizontal line between the Centre line and the goalpost line that is towards both end of the field it divides each halves of the field into 2 parts, or intentionally hits the ball over the backline of the goalpost.
To signal a penalty corner in field hockey, the referee points both arms horizontally and directly toward the goal of the offending team. When a Penalty Corner tag is at least 5 seconds before the Shot is taken by the attacking team player as start time, and 10 seconds after the action has ended as end time. A shot attempt on goal is when the attacking team takes a shot towards the defending team’s goalpost. There can be 3 results to that attempt: one is that the shot can be on target, the shot can be off target, or it can be a goal when you see a shot attempt towards the goal tag at least 5 seconds before the Shot is taken by the attacking team player as start time and 10 seconds after the action has ended as end time. There is a secondary category as Hockey Quarters which helps to determine which quarter of the match we are in and what is the current duration of the match that can be seen on the top left side of the screen this tag has to be associated while tagging with all the primary categories it means every time you tag an action from the primary category you need to attach the hockey quarters as a label to it.
Outcome
The model correctly identified the action, the team performing it, the outcome and the event boundaries. It successfully tagged four actions within a one-minute passage of play.
Learning
The largest improvement came from combining three types of instruction: what the event means, what it looks like on screen and when the event begins and ends. For sports-video analysis, visual and temporal guidance can be as important as the formal definition of the event.
Prompt 3
Prompt –
Two teams are playing a game of Field Hockey on a turf which is completely blue in colour. One of them is called Team A, and the other is called Team B. Team A is wearing a blue jersey and is attacking from left to right on the screen, and Team B is wearing White Jersey attacking from right to left on the field.
The start of any event must align with the initiation of one of the 10 primary categories: Circle Area Entry Team A, Circle Area Entry Team B, Hockey Long Corner Team A, Hockey Long Corner Team B, Hockey Penalty Corner Team A, Hockey Penalty Corner Team B, Shot Attempt Team A, Shot Attempt Team B, Turnover Team A, Turnover Team B. Please ignore moments of no action, such as injury breaks, discussions with the referee, or when the ball is out of play. The end of an event occurs when the primary category no longer matches the current action or the phase of play shifts.
Circle entry is when a player of either the blue team or the white team enters the Semicircle area that appears in front of the goalpost by a solid white line. When there is a circle entry, you will see that the defending team will be very compact and in front of the goalpost. There is a secondary category, Circle Area Entry Result, which tells us about the next action that has happened after a player from either of the teams entered the area, which further has labels, which means the outcome of the Circle Area Entry, i.e., Shot On Target, Goal, Shot Target. A Shot On Target is when the player in possession of the ball, i.e., the attacking team, takes a shot towards the defending team’s goalpost and the defending team’s Goalkeeper saves it from going inside the goalpost. A Goal is when the Player from the attacking team takes a shot towards the opponent’s goalpost, and the opponent’s goalkeeper is not able to save the ball, and it goes inside the goalpost. A Shot Off Target is when the player of the attacking team takes a shot, but it doesn’t go near the defending team’s goalpost or is blocked by any of the opponent’s players other than the goalkeeper. When you see a Circle Area Entry tag at least 5 seconds before the entry as start time and 10 seconds after the action is completed as end time. A long corner is awarded when the ball goes over the defending team’s backline that is behind the goalpost line and was last touched by a defending player, provided it was not an intentional clearance, also the signal given by the referee that is on the field wearing a yellow colour jersey will signal by raising one arm horizontally and pointing with it directly toward the corner flag on the side of the field where the ball went out over the backline when you see a Long Corner tag at least 5 seconds before the pass is being made by the attacking team player from the corners as start time and 10 seconds after the action ends as end time. A Hockey penalty corner is awarded to the team in possession of the ball when the defending team player commits a foul inside the circle or makes an intentional foul inside the 23-meter area to identify the 23-meter area we should look at the solid white horizontal line between the Centre line and the goalpost line that is towards both end of the field it divides each halves of the field into 2 parts, or intentionally hits the ball over the backline of the goalpost.
To signal a penalty corner in field hockey, the referee points both arms horizontally and directly toward the goal of the offending team. When a Penalty Corner tag is at least 5 seconds before the Shot is taken by the attacking team player as start time, and 10 seconds after the action has ended as end time. A shot attempt on goal is when the attacking team takes a shot towards the defending team’s goalpost. There can be 3 results to that attempt: one is that the shot can be on target, the shot can be off target, or it can be a goal when you see a shot attempt towards the goal tag at least 5 seconds before the Shot is taken by the attacking team player as start time and 10 seconds after the action has ended as end time. Turnover is when the player who is in possession of the ball losses the ball and the opponent team gets in possession of the ball and makes an attack towards the goalpost of the team which lost the ball Turnover can happen in 2 ways one is Miss pass that is when the player in possession of the ball tries to pass the ball to his own teammate wearing the same jersey colour but miss passes it to his opponent team player who is wearing a different jersey colour. The other way of a turnover is when the player in possession of the ball tries to pass the ball to his own teammate, wearing the same jersey colour, but the opponent player, who is wearing a different jersey colour, comes in between the pass and snatches the ball and starts to attack towards the goalpost of the team which lost the ball there is a secondary category attached to it as the Turnover Area which means in which part of the field did the turnover happen was it in the own half that is the half of the field the team is defending the goal or in the opponent half where the team is trying to score the goal. There is a secondary category as Hockey Quarters which helps to determine which quarter of the match we are in and what is the current duration of the match that can be seen on the top left side of the screen this tag has to be associated while tagging with all the primary categories it means every time you tag an action from the primary category you need to attach the hockey quarters as a label to it.
Outcome
The model was able to identify the action, the team, the field area and the outcome across continuous passages of play. However, a sideline restart was incorrectly tagged as a long corner. Some turnover tags began only when the opponent received the ball instead of including the build-up, and two entries into the dotted semicircle were incorrectly tagged as genuine circle entries.
Learning
The AI needed negative instructions and exclusion rules, not only positive event definitions. The next prompt had to explain which semicircle should not count, which restart locations should not be treated as long corners and how early a turnover tag should begin. Similar-looking events must be explicitly separated in the prompt.
Prompt 4
Prompt –
Two teams are playing a game of Field Hockey on a turf which is completely blue in colour. One of them is called Team A, and the other is called Team B. Team A is wearing a blue jersey and is attacking from left to right on the screen, and Team B is wearing White Jersey attacking from right to left on the field.
The start of any event must align with the initiation of one of the 10 primary categories: Circle Area Entry Team A, Circle Area Entry Team B, Hockey Long Corner Team A, Hockey Long Corner Team B, Hockey Penalty Corner Team A, Hockey Penalty Corner Team B, Shot Attempt Team A, Shot Attempt Team B, Turnover Team A, Turnover Team B. Please ignore moments of no action, such as injury breaks, discussions with the referee, or when the ball is out of play. The end of an event occurs when the primary category no longer matches the current action or the phase of play shifts.
Circle entry is when a player of either the blue team or the white team enters the Semicircle area that appears in front of the goalpost by a solid white line. There will also be a white dotted semicircle, which also appears in front of the goalpost. Do Not Include that. When there is a circle entry, you will see that the defending team will be very compact and in front of the goalpost. There is a secondary category, Circle Area Entry Result, which tells us about the next action that has happened after a player from either of the teams entered the area, which further has labels, which means the outcome of the Circle Area Entry, i.e., Shot On Target, Goal, Shot Target. A Shot On Target is when the player in possession of the ball, i.e., the attacking team, takes a shot towards the defending team’s goalpost and the defending team’s Goalkeeper saves it from going inside the goalpost. A Goal is when the Player from the attacking team takes a shot towards the opponent’s goalpost, and the opponent’s goalkeeper is not able to save the ball, and it goes inside the goalpost. A Shot Off Target is when the player of the attacking team takes a shot, but it doesn’t go near the defending team’s goalpost or is blocked by any of the opponent’s players other than the goalkeeper. When you see a Circle Area Entry tag at least 5 seconds before the entry as start time and 10 seconds after the action is completed as end time. A long corner is awarded when the ball goes over the defending team’s backline that is behind the goalpost line and was last touched by a defending player, provided it was not an intentional clearance, also the signal given by the referee that is on the field wearing a yellow colour jersey will signal by raising one arm horizontally and pointing with it directly toward the corner flag on the side of the field where the ball went out over the backline a long corner will only be taken from the exact corner spot on the field that will have flags standing.Do not include anything as a Long Corner if it is taken from the Outside line, which is the line which marks the area of the play when you see a Long Corner tag at least 5 seconds before the pass is being made by the attacking team player from the corners, as start time and 10 seconds after the action ends as end time. A Hockey penalty corner is awarded to the team in possession of the ball when the defending team player commits a foul inside the circle or makes an intentional foul inside the 23-meter area to identify the 23-meter area we should look at the solid white horizontal line between the Centre line and the goalpost line that is towards both end of the field it divides each halves of the field into 2 parts, or intentionally hits the ball over the backline of the goalpost. To signal a penalty corner in field hockey, the referee points both arms horizontally and directly toward the goal of the offending team. When a Penalty Corner tag is at least 5 seconds before the Shot is taken by the attacking team player as start time, and 10 seconds after the action has ended as end time. A shot attempt on goal is when the attacking team takes a shot towards the defending team’s goalpost.
There can be 3 results to that attempt: one is that the shot can be on target, the shot can be off target, or it can be a goal when you see a shot attempt towards the goal tag at least 5 seconds before the Shot is taken by the attacking team player as start time and 10 seconds after the action has ended as end time. Turnover is when the player who is in possession of the ball losses the ball and the opponent team gets in possession of the ball and makes an attack towards the goalpost of the team which lost the ball Turnover can happen in 2 ways one is Miss pass that is when the player in possession of the ball tries to pass the ball to his own teammate wearing the same jersey colour but miss passes it to his opponent team player who is wearing a different jersey colour. The other way of a turnover is when the player in possession of the ball tries to pass the ball to his own teammate, wearing the same jersey colour, but the opponent player, who is wearing a different jersey colour, comes in between the pass and snatches the ball and starts to attack towards the goalpost of the team which lost the ball there is a secondary category attached to it as the Turnover Area which means in which part of the field did the turnover happen was it in the own half that is the half of the field the team is defending the goal or in the opponent half where the team is trying to score the goal when you see a turnover tag at least 5 seconds before the ball is being snatched by the opponent player from the player in possession of the ball as start time and 10 seconds after the action has ended as end time.. There is a secondary category as Hockey Quarters which helps to determine which quarter of the match we are in and what is the current duration of the match that can be seen on the top left side of the screen this tag has to be associated while tagging with all the primary categories it means every time you tag an action from the primary category you need to attach the hockey quarters as a label to it.
Outcome
The AI correctly identified several existing KPIs, including circle entries created through long aerial passes, turnovers, penalty corners and shot attempts. However, it still had difficulty distinguishing some restarts near the side or backline from long corners. It also created two duplicate tags.
Learning
Explicit exclusions reduced several false positives, but difficult edge cases still required further refinement. The next improvements should focus on distinguishing corner restarts from other backline or sideline restarts, preventing duplicate detections and continuing to validate AI-generated tags before using them in reports. The experiment showed that prompt development works best as a continuous feedback loop rather than a one-time setup.
