The problem is that a tome takes 2 slots in your inventory, which you may want to keep for charms...
For the OP, do you use the scrolls by clicking on it or pressing the number corresponding to the line of scrolls in the belt ?
I press the number.
For instance, I tend to keep my scrolls of identify in the 3rd slot. When I get an item worth identifying (I'm very picky), I pick it up, press '3', which then opens my inventory, and left click on the item. The whole process takes mere seconds.
The problem is that the next scroll in line does not drop down to the lowest position on the belt. I suppose I could just quit whining and use the mouse though...