Understanding the score

The score in its pure form is just a number, and does not help us understand the make-up of the solution it represents. It doesn’t say which constraints are broken, or what caused them to break. To understand the score, it needs to be broken down.

The easiest way to do that during development is to print the score summary:

Java
Python

SolutionManager<VehicleRoutePlan, HardSoftLongScore> solutionManager = SolutionManager.create(solverFactory);
System.out.println(solutionManager.explain(vehicleRoutePlan));

solution_manager = SolutionManager.create(solver_factory)
print(solution_manager.explain(vehicle_route_plan))

For example, in vehicle routing, this prints that vehicle vehicle '1' matched the soft constraint minimizeTravelTime:

Explanation of score (0hard/-10343soft):
    Constraint matches:
        -10343soft: constraint (minimizeTravelTime) has 6 matches:
            -2335soft: justified with (MinimizeTravelTimeJustification[vehicleName=1, totalDrivingTimeSeconds=2335, description=Vehicle '1' total travel time is 0 hours 39 minutes.])
            ...
        ...
    Indictments (top 5 of 6):
        -2335soft: indicted with (1) has 1 matches:
            -2335soft: constraint (minimizeTravelTime)
        ...

Do not attempt to parse this string or expose it in services. It serves for debugging purposes only. Use score analysis instead.

In the string above, there are some previously unexplained concepts.

A Constraint match is created every time a constraint causes a change to the score.
Justifications are user-defined objects that implement the ai.timefold.solver.core.api.score.stream.ConstraintJustification interface, which carry meaningful information about a constraint match, such as its name and any metadata that the user chooses to expose. Justifications are most easily available via score analysis.
Indicted objects are objects which were directly involved in causing a constraint to match. For example, if your constraints penalize each vehicle, then there will be one ai.timefold.solver.core.api.score.constraint.Indictment instance per vehicle, carrying the vehicle as an indicted object. Indictments are typically used for heat map visualization.

Constraint Streams API can analyze the score natively. Incremental Java score calculation requires implementing an extra interface. Easy Java score calculation does not support score explanation.

1. Score Analysis: Which constraints are broken?

If other parts of your application, such as your web interface, need to describe the solution to the user, use the SolutionManager API:

Java
Python

SolutionManager<VehicleRoutePlan, HardSoftLongScore> solutionManager = SolutionManager.create(solverFactory);
ScoreAnalysis<HardSoftLongScore> scoreAnalysis = solutionManager.analyze(vehicleRoutePlan);

solution_manager = SolutionManager.create(solver_factory)
score_analysis = solution_manager.analyze(vehicle_route_plan)

ScoreAnalysis is a JSON-friendly representation of the score, breaking down the score into individual constraints. Using score analysis, you can find out:

What’s the total score.
Which constraints are broken, and how many times.
Which planning entities and problem facts are responsible for breaking which constraints.

For performance reasons and especially with large datasets that you’ll later need to serialize, you may choose to use ScoreAnalysis without justifications, while still maintaining the count of constraint matches. In that case, use ScoreAnalysisFetchPolicy.FETCH_MATCH_COUNT instead of the default ScoreAnalysisFetchPolicy.FETCH_ALL when calling SolutionManager.analyze(…).

It is also possible to print the score summary:

Java
Python

SolutionManager<VehicleRoutePlan, HardSoftLongScore> solutionManager = SolutionManager.create(solverFactory);
ScoreAnalysis<HardSoftLongScore> scoreAnalysis = solutionManager.analyze(vehicleRoutePlan);
System.out.println(scoreAnalysis.summarize());

solution_manager = SolutionManager.create(solver_factory)
score_analysis = solution_manager.analyze(vehicle_route_plan)
print(score_analysis.summary)

For example, this prints that vehicle vehicle '1' matched the soft constraint minimizeTravelTime:

Explanation of score (0hard/-10343soft):
    Constraint matches:
        -10343soft: constraint (minimizeTravelTime) has 6 matches:
            -2335soft: justified with (MinimizeTravelTimeJustification[vehicleName=1, totalDrivingTimeSeconds=2335, description=Vehicle '1' total travel time is 0 hours 39 minutes.])
            ...
        ...

Do not attempt to parse this string or expose it in services. It serves for debugging purposes only.

1.1. Finding the broken constraints

When you have the ScoreAnalysis instance, you can find out which constraints are broken:

Java
Python

scoreAnalysis.constraintMap().forEach((constraintRef, constraintAnalysis) -> {
   String constraintId = constraintRef.constraintId();
   HardSoftScore scorePerConstraint = constraintAnalysis.score();
   ...
});

for constraint_ref, constraint_analysis in score_analysis.constraint_map.items():
   constraint_id = constraint_ref.constraint_id
   score_per_constraint = constraint_analysis.score
   ...

If you wish to go further and find out which planning entities and problem facts are responsible for breaking the constraint, you can further explore the ConstraintAnalysis instance you received from ScoreAnalysis.constraintMap():

Java
Python

int matchCount = constraintAnalysis.matchCount();
constraintAnalysis.matches().forEach(matchAnalysis -> {
    HardSoftScore scorePerMatch = matchAnalysis.score();
    Object justification = matchAnalysis.justification();
    ...
});

match_count = constraint_analysis.match_count

for match_analysis in constraint_analysis.matches:
    score_per_match = match_analysis.score
    justification = match_analysis.justification
    ...

Each match is accompanied by the score difference it caused, and a justification object (see above). Typically, the scoring engine creates justification objects automatically by using the results of Constraint Streams' justifyWith(…) call.

1.2. Identifying changes between two solutions

If you have two different solutions from the Solver, you can compare them using ScoreAnalysis and find out what changed between them:

Java
Python

ScoreAnalysis<HardSoftScore>  firstAnalysis = solutionManager.analyze(firstSolution);
ScoreAnalysis<HardSoftScore> secondAnalysis = solutionManager.analyze(secondSolution);
ScoreAnalysis<HardSoftScore>           diff = firstAnalysis.diff(secondAnalysis);

// Score difference only carries the constraints whose matches changed from first to second solution.
diff.constraintMap().forEach((constraintRef, constraintAnalysis) -> {
   String constraintId = constraintRef.constraintId();
   HardSoftScore scoreDiff = constraintAnalysis.score();
   // Matches only include constraint matches that:
   //   - the second solution either added to or removed from the first solution.
   //   - had their score changed.
   // Two matches are considered equal if their justification objects are equal.
   constraintAnalysis.matches().forEach(matchAnalysis -> {
       ...
   });
});

first_analysis = solution_manager.analyze(firstSolution)
second_analysis = solution_manager.analyze(second_solution)
diff = first_analysis - second_analysis

# Score difference only carries the constraints whose matches changed from first to second solution.
for constraint_ref, constraint_analysis in diff.constraint_map.items():
    constraint_id = constraint_ref.constraint_id
    score_diff = constraint_analysis.score
    # Matches only include constraint matches that:
    #   - the second solution either added to or removed from the first solution.
    #   - had their score changed.
    # Two matches are considered equal if their justification objects are equal.
    for match_analysis in constraint_analysis.matches:
        ...

Think of diff(…) as a subtraction operation, where the second solution is subtracted from the first solution. For example, if the first solution has score of 2hard/3soft and the second solution has score of 1hard/2soft, then the score difference will be 1hard/1soft, indicating that the second solution is better than the first solution.

The same applies to constraints and constraint matches. If a constraint did not match in the first solution but did match in the second, then the constraint match will be included in the diff as negative. If instead the constraint did match in the first solution but did not match in the second, then the constraint match will be included in the diff as positive.

1.3. Sending score analysis over the wire

The purpose of ScoreAnalysis is to break down the score so that the end user can understand it. To succeed at this, ScoreAnalysis is JSON-friendly and can be easily sent over the wire from the backend to the frontend.

ScoreAnalysis instances will serialize into JSON automatically (using Jackson):

If you use Timefold Solver’s Quarkus integration,
or if you use Timefold Solver’s Spring Boot integration,
or if you directly included the timefold-solver-jackson module in your project.

If you implemented ConstraintJustication to provide custom justification objects, you are responsible for making them JSON-friendly yourself.

ScoreAnalysis doesn’t natively deserialize from JSON back to Java objects. This is because we have no way of knowing which Score or ConstraintJustification implementations you may be using. However, deserialization is easy to implement yourself by extending AbstractScoreAnalysisJacksonDeserializer and registering it with Jackson’s ObjectMapper.

With large datasets, you may choose to use ScoreAnalysis without justifications, while still maintaining the count of constraint matches. In that case, use ScoreAnalysisFetchPolicy.FETCH_MATCH_COUNT instead of the default ScoreAnalysisFetchPolicy.FETCH_ALL when calling SolutionManager.analyze(…).

2. Solution Diff: What changed between now and then?

The solution diff is available as a preview feature and must be specifically enabled in the solver configuration.

This feature is not yet supported in Timefold Solver for Python.

Using the SolutionManager API, you can compare two solutions provided by the solver, and find out what changed between them:

Java

SolutionManager<VehicleRoutePlan, HardSoftLongScore> solutionManager = SolutionManager.create(solverFactory);
...
PlanningSolutionDiff<VehicleRoutePlan> solutionDiff = solutionManager.diff(oldSolution, newSolution);

The PlanningSolutionDiff instance contains the following information:

A full list of entities whose planning variables (genuine or shadow) changed between the two solutions. Each entity is represented by a PlanningEntityDiff instance, which contains the entity itself, as well as the old and new values of the changed variables.
Set of planning entities not present in the new solution (removed entities).
Set of planning entities not present in the old solution (added entities).
The oldSolution and newSolution planning solutions.

It also has a useful toString() for a quick overview of the changes. Do not attempt to parse this string or expose it in services, its format is not stable and is subject to change.

3. Heat map: Visualize the hot planning entities

To show a heat map in the UI that highlights the planning entities and problem facts have an impact on the Score, get the Indictment map from the ScoreExplanation:

Java
Python

SolutionManager<VehicleRoutePlan, HardSoftLongScore> solutionManager = SolutionManager.create(solverFactory);
ScoreExplanation<VehicleRoutePlan, HardSoftLongScore> scoreExplanation = solutionManager.explain(vehicleRoutePlan);
Map<Object, Indictment<HardSoftLongScore>> indictmentMap = scoreExplanation.getIndictmentMap();
for (Visit visit : vehicleRoutePlan.getVisits()) {
    Indictment<HardSoftLongScore> indictment = indictmentMap.get(visit);
    if (indictment == null) {
        continue;
    }
    // The score impact of that planning entity
    HardSoftLongScore totalScore = indictment.getScore();

    for (ConstraintMatch<HardSoftLongScore> constraintMatch : indictment.getConstraintMatchSet()) {
        String constraintName = constraintMatch.getConstraintName();
        HardSoftLongScore score = constraintMatch.getScore();
        ...
    }
}

solution_manager = SolutionManager.create(solver_factory)
score_explanation = solution_manager.explain(vehicle_route_plan)
indictment_map = score_explanation.indictment_map
for visit in vehicle_route_plan.visits:
    indictment = indictment_map.get(visit)
    if indictment is None:
        continue
    # The score impact of that planning entity
    total_score = indictment.score

    for constraint_match in indictment.constraint_match_set:
        constraint_name = constraint_match.constraint_name
        score = constraint_match.score
        ...

ScoreExplanation should only be used for processing indictments. For analyzing the score and processing constraint matches, use score analysis instead, which is faster and JSON-friendly.

Each Indictment is the sum of all constraints where that justification object is involved with. The sum of all the Indictment.getScoreTotal() differs from the overall score, because multiple Indictments can share the same ConstraintMatch.