Do you have DrawOnInventoryUpdate set to true?
Also if you are enabling, disableing panels play around with the DrawOnEnable
The disaplay Panel binding also has some settings for drawing the container:
You can also choose to draw it through code for more control over when it happens