Bookmarks

🟢 Easy

Arithmetic Operators

What: Arithmetic operators are the calculators of Python — + - * / for basics, % (modulus) for remainders, // (floor division) for whole batches, and ** for powers. Why it matters: In PySpark/Data Engineering, % decides partition assignment (hash(key) % numPartitions = even/odd or skew check), while // computes how many full batches fit (total_rows // batch_size without a leftover float). How to remember: / gives a float (9.5), // chops the decimal (9), % keeps only the left-over (1).

Operators at a glance — with Data Engineering use
19 / 2
9.5
float division — exact
19 // 2
9
floor — batches of 2
13 % 2
1 → odd
remainder — partition key
2 ** 4
16
power — growth calc
Partition check: row_id % 2 == 0 ? even partition : odd  |  Batches: 1000 rows // 100 = 10 full batches
Python
print(1+1)     # 2 addition
print(10-3)    # 7
print(3*4)     # 12
print(19/2)    # 9.5 division (float)
print(19//2)   # 9 floor — round down
print(13%2)    # 1 modulus — remainder
print(2**4)    # 16 power
x = 11
print(x % 2 == 0)  # False — 11 is odd
x = 0
if x % 2 == 0:
    print('even')
else:
    print('odd')
Output
2
7
12
9.5
9
1
16
False
even
🟡 Intermediate

Comparison & Logical (truth tables added)

What: Comparison operators (== != > < >= <=) answer yes/no about values, while logical operators (and / or / not) combine those answers — the exact logic behind every df.filter((col('age') > 30) & (col('city') == 'bangalore')) in PySpark. Why it matters: A single wrong and vs or can silently drop half your rows; data-quality gates live or die on these truth tables. How to remember: and needs both True, or needs just one True, and is checks identity (same object) not just equality — use the visual table below as your cheat sheet.

Truth table visual — and / or / not
AND — both must be True
A
B
A and B
T
T
True ✓
T
F
False
F
F
False
OR — one True is enough
A
B
A or B
T
T
True ✓
T
F
True ✓
F
F
False
not True → False a == b values equal a is b same object ✗
Python
print('praveen'=='praveen')  # True
print(1==1, 1!=2, 5>3, 3<5, 5>=5, 3<=5)

# True and True  → True
# True and False → False
# False and False→ False
# True or False  → True
# False or False → False
x, y = 10, 20
print(x>10 or y==20)        # True
print(not(x>5 and y==20))   # False
print(not(x<5 or y==6))     # True

a = [1,2]; b = [1,2]
print(a == b)   # True — values equal
print(a is b)   # False — different objects
print(a is not b)  # True
Output
True
True True True True True True
True
False
True
True
False
True
🟢 Easy

Assignment Shortcuts

What: Assignment shortcuts like += -= *= /= %= are just shorthands — x += 5 means x = x + 5, x *= 2 means x = x * 2, but shorter and less error-prone. Why it matters: In Data Engineering you write accumulators constantly — total += row_value in a loop, retry_count *= 2 for exponential back-off, or offset %= partitions to wrap around. How to remember: The symbol before = is the operation you want to repeat on the same variable — read x += 5 as "add 5 to x, store back in x".

Shortcut unfolding — long form vs short form
long form
x = x + 5
x += 5
long form
x = x * 2
x *= 2
long form
x = x % 3
x %= 3
All shortcuts: += -= *= /= %= //= **= — same pattern, different operator.
Python
x = 10
x += 5; print(x)  # 15  (x = x+5)
x *= 2; print(x)  # 30  (x = x*2)
x -= 10; print(x) # 20
x /= 4; print(x)  # 5.0
x %= 3; print(x)  # 2.0
Output
15
30
20
5.0
2.0
🟡 Intermediate

Casting & Type Conversion

What: Casting converts one type to another — int(10.66) → 10 truncates, float('1') → 1.0 adds a decimal, str(10) → '10' makes it text. Why it matters: PySpark's spark.read.csv often infers every column as string, so before you aggregate you must cast — col('salary').cast('int') — and dirty rows like 'praveen' in a numeric column will fail just as int('praveen') throws ValueError here. How to remember: Think of casting as a gate: good strings pass ('1' → 1), bad strings are rejected ('praveen' → error), and the error itself is your data-quality signal.

Casting pipeline — CSV string → trusted type
CSV: '10.66'int('10.66') ✗float→int ✓ 10 | 'praveen' → int ✗ ValueErrorbad data → filter out
PySpark parallel: F.col('age').cast('int')  |  guard: try: int(v) except ValueError: mark as null
Python
print(int(10.66))    # 10 — truncates
print(float('1'))   # 1.0
print(str(10))      # '10'
print(int('1'))     # 1
print(bool(0), bool(1), bool(''))  # False True False

try:
    print(int('praveen'))
except ValueError as e:
    print('cannot cast:', e)
Output
10
1.0
10
1
False True False
cannot cast: invalid literal for int() with base 10: 'praveen'