Arithmetic Operators
What: Arithmetic operators are the calculators of Python — + - * / for basics, % (modulus) for remainders, // (floor division) for whole batches, and ** for powers. Why it matters: In PySpark/Data Engineering, % decides partition assignment (hash(key) % numPartitions = even/odd or skew check), while // computes how many full batches fit (total_rows // batch_size without a leftover float). How to remember: / gives a float (9.5), // chops the decimal (9), % keeps only the left-over (1).
print(1+1) # 2 addition
print(10-3) # 7
print(3*4) # 12
print(19/2) # 9.5 division (float)
print(19//2) # 9 floor — round down
print(13%2) # 1 modulus — remainder
print(2**4) # 16 power
x = 11
print(x % 2 == 0) # False — 11 is odd
x = 0
if x % 2 == 0:
print('even')
else:
print('odd')2
7
12
9.5
9
1
16
False
evenComparison & Logical (truth tables added)
What: Comparison operators (== != > < >= <=) answer yes/no about values, while logical operators (and / or / not) combine those answers — the exact logic behind every df.filter((col('age') > 30) & (col('city') == 'bangalore')) in PySpark. Why it matters: A single wrong and vs or can silently drop half your rows; data-quality gates live or die on these truth tables. How to remember: and needs both True, or needs just one True, and is checks identity (same object) not just equality — use the visual table below as your cheat sheet.
not True → False
a == b values equal
a is b same object ✗
print('praveen'=='praveen') # True
print(1==1, 1!=2, 5>3, 3<5, 5>=5, 3<=5)
# True and True → True
# True and False → False
# False and False→ False
# True or False → True
# False or False → False
x, y = 10, 20
print(x>10 or y==20) # True
print(not(x>5 and y==20)) # False
print(not(x<5 or y==6)) # True
a = [1,2]; b = [1,2]
print(a == b) # True — values equal
print(a is b) # False — different objects
print(a is not b) # TrueTrue
True True True True True True
True
False
True
True
False
TrueAssignment Shortcuts
What: Assignment shortcuts like += -= *= /= %= are just shorthands — x += 5 means x = x + 5, x *= 2 means x = x * 2, but shorter and less error-prone. Why it matters: In Data Engineering you write accumulators constantly — total += row_value in a loop, retry_count *= 2 for exponential back-off, or offset %= partitions to wrap around. How to remember: The symbol before = is the operation you want to repeat on the same variable — read x += 5 as "add 5 to x, store back in x".
+= -= *= /= %= //= **= — same pattern, different operator.x = 10
x += 5; print(x) # 15 (x = x+5)
x *= 2; print(x) # 30 (x = x*2)
x -= 10; print(x) # 20
x /= 4; print(x) # 5.0
x %= 3; print(x) # 2.015
30
20
5.0
2.0Casting & Type Conversion
What: Casting converts one type to another — int(10.66) → 10 truncates, float('1') → 1.0 adds a decimal, str(10) → '10' makes it text. Why it matters: PySpark's spark.read.csv often infers every column as string, so before you aggregate you must cast — col('salary').cast('int') — and dirty rows like 'praveen' in a numeric column will fail just as int('praveen') throws ValueError here. How to remember: Think of casting as a gate: good strings pass ('1' → 1), bad strings are rejected ('praveen' → error), and the error itself is your data-quality signal.
F.col('age').cast('int') | guard: try: int(v) except ValueError: mark as nullprint(int(10.66)) # 10 — truncates
print(float('1')) # 1.0
print(str(10)) # '10'
print(int('1')) # 1
print(bool(0), bool(1), bool('')) # False True False
try:
print(int('praveen'))
except ValueError as e:
print('cannot cast:', e)10
1.0
10
1
False True False
cannot cast: invalid literal for int() with base 10: 'praveen'