A Critical Hit is an attack that does more than the usual amount of damage to a monster.
When an attack scores a critical hit, a different animation is used instead of the normal hit animation.
The minimum critical hit rate from DEX vs. AGI is 5%, the maximum is 20%. Equipment and merits that increase critical hit rate are added on top of this.
Critical Hit rate follows an exponential curve. Minor increases in DEX will not have a significant impact on critical hit rate (e.g. DEX+1 = +0.25 critical hit rate). Major increases in DEX increase the rate significantly, whereby DEX+1 ~= +1% critical hit rate.