Barrel trimmer is a great tool to make pen barrels true square. Just a little bit of shaving will be fine. The trimmer tip can remove dry glue efficiently.
You can use it by bare hand or drill press, depending on the length you feel like to shave.
For larger barrels, just turn a wooden spacer by 7 or 8mm brass tubes.
7mm trimmer cutter head is 16mm OD
8mm trimmer cutter head is 19.5mm OD