Creating Views
Temporary views allow you to run SQL queries on DataFrames. They are like temporary tables that exist only for the duration of the Spark session.
Creating a Temporary View
df.createOrReplaceTempView("my_table")Global Temporary View
A global view is accessible across all Spark sessions and lives in the `global_temp` database.
df.createOrReplaceGlobalTempView("global_view")
spark.sql("SELECT * FROM global_temp.global_view").show()Why Use Views?
- If you are comfortable with SQL, you can manipulate data using SQL syntax.
- Views are lazily evaluated – they don’t materialise data until queried.
- Useful for complex joins and aggregations expressed more clearly in SQL.
View Lifetime
Temporary views are dropped when the Spark session ends. Global views persist across sessions but are removed when the application terminates.
Two Minute Drill
- `createOrReplaceTempView()` creates a session‑scoped SQL view.
- `createOrReplaceGlobalTempView()` creates a cross‑session view.
- Use SQL queries with `spark.sql()` after creating views.
- Views are lazy – no data movement until query.
Need more clarification?
Drop us an email at career@quipoinfotech.com