In binary files, a single byte is often used to represent numbers from 0 to 255 (unsigned). However, in Java, a byte is signed, ranging from -128 to 127, because Java doesn't have a unsigned types (with the exception of the 2 byte char type). A raw byte with value 0xF0 in a file meant to represent 240 will become -16 when using Javas's ByteBuffer.get(). To fix this:
int unsignedByte = buffer.get() & 0xFF;
Explanation: When performing bitwise operations, Java automatically "promotes" the 8-bit byte to a 32-bit signed integer. If the byte is 0xF0 = 240 (11110000), Java sees the leading 1 and assumes it is a negative number. Through Sign Extension, it fills the new 24 bits with 1s to preserve that negative value (-16) in the larger container.
Original Byte: 11110000 (-16, see two's complement) Promoted Int : 11111111 11111111 11111111 11110000 (Still -16)Now, you apply the mask 0xFF (255). In binary, 0xFF as a 32-bit integer is 00000000 00000000 00000000 11111111:
11111111 11111111 11111111 11110000 (The promoted -16) & 00000000 00000000 00000000 11111111 (The 0xFF mask) ------------------------------------- 00000000 00000000 00000000 11110000 (The result: 240)
By "ANDing" the promoted integer with 0xFF, you effectively clear out all the 1s created by sign extension, leaving only the original 8 bits and giving you the correct unsigned value of 240.
Similary, reading an unsigned short (16-bit) from file:
int unsignedShort = buffer.getShort() & 0xFFFF;
Reading an unsigned int (32-bit):
long unsignedInt = buffer.getInt() & 0xFFFFFFFFL;
Note that an unsigned 32-bit integer can exceed the capacity of a Java int. You must jump up to a long and use a long literal mask (noted by the L at the end of the mask value).
When writing a value that is first multplied by a scale factor of 2^31 (1 << 31), Java assumes the 1 << 31 is an int and shifting 1 by 31 places puts it into the sign bit position of a 32bit int, which results in the negative value of -2147483648. Correct usage:
double val = 123;
long scaleFactor = (1L << 31); // Will be positive 2147483648 because long is 64bit
// and shifting by 31 won't put the 1 into sign bit
// position
long val_scaled = (long) (val * scaleFactor);