What are applications, jobs, stages and tasks in Spark?
September 29, 2021What is the difference between foreach and foreachPartition in Spark?
October 4, 2021Simple problem with a simple solution.
Solution
Use the powerful regexp_replace function to replace characters.
regexp_replace(string INITIAL_STRING, string PATTERN, string REPLACEMENT):
Replace tab in the string with an empty string
select regexp_replace(description, '\\t', '') as description
Replace special characters in the string with empty character
select regexp_replace(description, ‘[^0-9a-zA-Z]+’, ‘’) as description
Big Data In Real World
We are a group of Big Data engineers who are passionate about Big Data and related Big Data technologies. We have designed, developed, deployed and maintained Big Data applications ranging from batch to real time streaming big data platforms. We have seen a wide range of real world big data problems, implemented some innovative and complex (or simple, depending on how you look at it) solutions.
