55 tasks • Based on golden solution embeddings • Click any cell to compare
| Merge Conflict Resolution | task_1 | task_6 | task_8 | task_9 | task_13 | task_14 | task_24 | task_33 | task_35 | task_38 | task_48 | task_58 | task_63 | task_68 | task_73 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| task_1 | - | 0.85 | 0.79 | 0.83 | 0.56 | 0.61 | 0.70 | 0.58 | 0.44 | 0.55 | 0.66 | 0.63 | 0.63 | 0.57 | 0.66 |
| task_6 | 0.85 | - | 0.79 | 0.86 | 0.49 | 0.60 | 0.73 | 0.56 | 0.37 | 0.58 | 0.65 | 0.62 | 0.67 | 0.50 | 0.61 |
| task_8 | 0.79 | 0.79 | - | 0.77 | 0.43 | 0.53 | 0.79 | 0.65 | 0.36 | 0.67 | 0.77 | 0.56 | 0.65 | 0.46 | 0.55 |
| task_9 | 0.83 | 0.86 | 0.77 | - | 0.43 | 0.65 | 0.71 | 0.55 | 0.41 | 0.56 | 0.64 | 0.68 | 0.69 | 0.61 | 0.67 |
| task_13 | 0.49 | 0.43 | 0.43 | 0.43 | - | 0.48 | 0.38 | 0.36 | 0.53 | 0.33 | 0.36 | 0.49 | 0.45 | 0.33 | 0.37 |
| task_14 | 0.61 | 0.60 | 0.53 | 0.65 | 0.48 | - | 0.49 | 0.39 | 0.46 | 0.39 | 0.47 | 0.51 | 0.53 | 0.49 | 0.58 |
| task_24 | 0.70 | 0.73 | 0.79 | 0.71 | 0.38 | 0.49 | - | 0.72 | 0.30 | 0.71 | 0.80 | 0.55 | 0.59 | 0.46 | 0.53 |
| task_33 | 0.58 | 0.56 | 0.65 | 0.55 | 0.36 | 0.39 | 0.72 | - | 0.39 | 0.82 | 0.81 | 0.55 | 0.53 | 0.42 | 0.44 |
| task_35 | 0.44 | 0.37 | 0.36 | 0.41 | 0.53 | 0.46 | 0.30 | 0.39 | - | 0.34 | 0.35 | 0.48 | 0.45 | 0.40 | 0.39 |
| task_38 | 0.55 | 0.58 | 0.67 | 0.56 | 0.33 | 0.39 | 0.71 | 0.82 | 0.34 | - | 0.80 | 0.54 | 0.57 | 0.43 | 0.46 |
| task_48 | 0.66 | 0.65 | 0.77 | 0.64 | 0.36 | 0.47 | 0.80 | 0.81 | 0.35 | 0.80 | - | 0.54 | 0.58 | 0.43 | 0.52 |
| task_58 | 0.63 | 0.62 | 0.56 | 0.68 | 0.49 | 0.51 | 0.55 | 0.55 | 0.48 | 0.54 | 0.54 | - | 0.80 | 0.58 | 0.59 |
| task_63 | 0.63 | 0.67 | 0.65 | 0.69 | 0.48 | 0.53 | 0.59 | 0.53 | 0.45 | 0.57 | 0.58 | 0.80 | - | 0.57 | 0.63 |
| task_68 | 0.57 | 0.50 | 0.46 | 0.61 | 0.33 | 0.44 | 0.46 | 0.42 | 0.40 | 0.43 | 0.43 | 0.58 | 0.57 | - | 0.77 |
| task_73 | 0.66 | 0.61 | 0.55 | 0.67 | 0.37 | 0.58 | 0.53 | 0.44 | 0.39 | 0.46 | 0.52 | 0.59 | 0.63 | 0.77 | - |
| Interactive Rebase | task_2 | task_10 | task_11 | task_12 | task_15 | task_20 | task_25 | task_30 | task_34 | task_39 | task_44 | task_49 | task_59 | task_64 | task_69 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| task_2 | - | 0.60 | 0.66 | 0.63 | 0.66 | 0.61 | 0.69 | 0.60 | 0.67 | 0.66 | 0.69 | 0.69 | 0.58 | 0.58 | 0.67 |
| task_10 | 0.60 | - | 0.56 | 0.55 | 0.48 | 0.43 | 0.57 | 0.44 | 0.56 | 0.55 | 0.71 | 0.70 | 0.43 | 0.43 | 0.57 |
| task_11 | 0.66 | 0.56 | - | 0.61 | 0.49 | 0.62 | 0.55 | 0.54 | 0.59 | 0.58 | 0.57 | 0.58 | 0.61 | 0.61 | 0.69 |
| task_12 | 0.63 | 0.55 | 0.61 | - | 0.66 | 0.61 | 0.69 | 0.71 | 0.62 | 0.62 | 0.47 | 0.46 | 0.60 | 0.61 | 0.51 |
| task_15 | 0.66 | 0.48 | 0.49 | 0.66 | - | 0.48 | 0.75 | 0.57 | 0.75 | 0.76 | 0.53 | 0.52 | 0.46 | 0.46 | 0.51 |
| task_20 | 0.61 | 0.46 | 0.62 | 0.61 | 0.48 | - | 0.50 | 0.67 | 0.53 | 0.53 | 0.53 | 0.53 | 0.88 | 0.89 | 0.57 |
| task_25 | 0.69 | 0.57 | 0.55 | 0.69 | 0.75 | 0.50 | - | 0.68 | 0.86 | 0.81 | 0.64 | 0.64 | 0.53 | 0.53 | 0.55 |
| task_30 | 0.60 | 0.44 | 0.54 | 0.71 | 0.57 | 0.67 | 0.66 | - | 0.69 | 0.71 | 0.44 | 0.44 | 0.62 | 0.63 | 0.48 |
| task_34 | 0.67 | 0.56 | 0.59 | 0.71 | 0.75 | 0.53 | 0.86 | 0.69 | - | 0.92 | 0.61 | 0.60 | 0.53 | 0.53 | 0.57 |
| task_39 | 0.73 | 0.55 | 0.58 | 0.69 | 0.76 | 0.53 | 0.81 | 0.71 | 0.92 | - | 0.58 | 0.58 | 0.50 | 0.51 | 0.57 |
| task_44 | 0.69 | 0.71 | 0.57 | 0.47 | 0.53 | 0.53 | 0.64 | 0.44 | 0.61 | 0.58 | - | 0.99 | 0.59 | 0.58 | 0.76 |
| task_49 | 0.69 | 0.70 | 0.58 | 0.46 | 0.52 | 0.53 | 0.64 | 0.44 | 0.60 | 0.58 | 0.99 | - | 0.59 | 0.57 | 0.76 |
| task_59 | 0.58 | 0.46 | 0.61 | 0.60 | 0.46 | 0.88 | 0.53 | 0.62 | 0.53 | 0.50 | 0.59 | 0.59 | - | 0.98 | 0.57 |
| task_64 | 0.58 | 0.46 | 0.61 | 0.61 | 0.46 | 0.89 | 0.53 | 0.63 | 0.53 | 0.51 | 0.58 | 0.57 | 0.98 | - | 0.57 |
| task_69 | 0.67 | 0.65 | 0.69 | 0.51 | 0.51 | 0.57 | 0.55 | 0.48 | 0.57 | 0.57 | 0.76 | 0.76 | 0.57 | 0.57 | - |
| Recovery Tasks | task_18 | task_23 | task_28 | task_32 | task_37 | task_47 | task_50 | task_52 | task_57 | task_62 | task_77 |
|---|---|---|---|---|---|---|---|---|---|---|---|
| task_18 | - | 0.48 | 0.48 | 0.48 | 0.52 | 0.48 | 0.41 | 0.48 | 0.47 | 0.47 | 0.43 |
| task_23 | 0.48 | - | 0.89 | 0.85 | 0.82 | 0.55 | 0.36 | 0.54 | 0.78 | 0.54 | 0.81 |
| task_28 | 0.52 | 0.89 | - | 0.88 | 0.85 | 0.57 | 0.37 | 0.57 | 0.82 | 0.57 | 0.84 |
| task_32 | 0.52 | 0.85 | 0.88 | - | 0.83 | 0.56 | 0.37 | 0.56 | 0.80 | 0.55 | 0.81 |
| task_37 | 0.55 | 0.82 | 0.85 | 0.83 | - | 0.59 | 0.38 | 0.59 | 0.78 | 0.59 | 0.80 |
| task_47 | 0.48 | 0.55 | 0.57 | 0.56 | 0.59 | - | 0.50 | 1.00 | 0.58 | 0.99 | 0.57 |
| task_50 | 0.41 | 0.36 | 0.37 | 0.37 | 0.38 | 0.50 | - | 0.50 | 0.34 | 0.51 | 0.35 |
| task_52 | 0.48 | 0.54 | 0.57 | 0.56 | 0.59 | 1.00 | 0.50 | - | 0.57 | 0.99 | 0.57 |
| task_57 | 0.47 | 0.78 | 0.82 | 0.80 | 0.78 | 0.58 | 0.34 | 0.57 | - | 0.57 | 0.89 |
| task_62 | 0.47 | 0.54 | 0.57 | 0.55 | 0.59 | 0.99 | 0.50 | 0.99 | 0.57 | - | 0.56 |
| task_77 | 0.47 | 0.81 | 0.84 | 0.81 | 0.80 | 0.57 | 0.35 | 0.57 | 0.89 | 0.56 | - |
| Search Tasks | task_3 | task_22 | task_26 | task_27 | task_36 | task_66 | task_76 |
|---|---|---|---|---|---|---|---|
| task_3 | - | 0.42 | 0.88 | 0.44 | 0.41 | 0.32 | 0.40 |
| task_22 | 0.42 | - | 0.42 | 0.81 | 0.68 | 0.44 | 0.70 |
| task_26 | 0.88 | 0.42 | - | 0.44 | 0.39 | 0.33 | 0.39 |
| task_27 | 0.44 | 0.81 | 0.44 | - | 0.61 | 0.44 | 0.65 |
| task_36 | 0.41 | 0.68 | 0.39 | 0.61 | - | 0.43 | 0.49 |
| task_66 | 0.32 | 0.44 | 0.33 | 0.44 | 0.43 | - | 0.50 |
| task_76 | 0.40 | 0.70 | 0.39 | 0.65 | 0.49 | 0.50 | - |
| Iterative Committing | task_16 | task_21 | task_31 | task_45 | task_46 | task_56 | task_65 |
|---|---|---|---|---|---|---|---|
| task_16 | - | 0.80 | 0.73 | 0.67 | 0.28 | 0.33 | 0.81 |
| task_21 | 0.80 | - | 0.70 | 0.56 | 0.30 | 0.35 | 0.73 |
| task_31 | 0.73 | 0.70 | - | 0.63 | 0.33 | 0.37 | 0.79 |
| task_45 | 0.67 | 0.56 | 0.63 | - | 0.35 | 0.37 | 0.71 |
| task_46 | 0.28 | 0.30 | 0.33 | 0.35 | - | 0.48 | 0.35 |
| task_56 | 0.33 | 0.35 | 0.37 | 0.37 | 0.48 | - | 0.39 |
| task_65 | 0.81 | 0.73 | 0.79 | 0.71 | 0.35 | 0.39 | - |