machine learning - The output of a softmax isn't supposed to have zeros, right? -
i working on net in tensorflow produces vector passed through softmax output.
now have been testing , weirdly enough vector (the 1 passed through softmax) has zeros in coordinate one.
based on softmax's definition exponential, assumed wasn't supposed happen. error?
edit: vector 120x160 =192000. values float32
it may not error. need @ input softmax well. quite possible vector has negative values , single positive value. result in softmax output vector containing zeros , single 1 value.
you correctly pointed out softmax numerator should never have zero-values due exponential. however, due floating point precision, numerator small value, say, exp(-50000), evaluates zero.
Comments
Post a Comment