POS tagging a sentence in Scala using Spark NLP.
POS tagging is the process of marking up a word in a corpus to a corresponding part of a speech tag, based on its context and definition. This task is not straightforward, as
Apache Spark
Apache Spark is a unified analytics engine for big data processing, with built-in modules for streaming, SQL, machine learning and graph processing. this tags contains basic tutorial on various Apache spark functions
Basic encoding : label encoding and one hot encoding in Scala with Apache Spark
If your starting with machine learning, after cleaning the data you end up with Normalising data, this is where encoding techniques comes in handy. there are lot of data encoding techniques but we
4 Moments : Skew,Kurtosis,Mean,Variance in Scala and Apache Spark
Moments is specific quantitative measure of the shape of the data. In statistics, moments are used to understand the various characteristics of a probability distribution. usually we use moments to characterise the data,
Basic statistics concepts for machine learning in Scala spark
Before applying some distribution algorithm or probability density function or probability mass function, we need to understand some basic concepts of statistics these concepts might be though in our school ,we shall start
Removing Stop Words in Apache Spark using Scala
Long ago is was working on my pet project where i used scrape description and title form web URL and indexing words for granular search and grouping. the project was in java. and