Socio monitoring framework (SMF): Efficient sentiment analysis through informal and native terms

Article history: Received 2 February 2020 Received in revised form 29 July 2020 Accepted 8 August 2020 Prediction and analysis of public expression is the trending topic of the current research arena. Opinion mining (a.k.a. Sentiment Analysis) is the automated orientation of public sentiments, views, suggestions, and opinions. It assists in estimating the popularity of products, events, services, and even political policies via user-generated content. Machine learning based supervised, semi-supervised, and unsupervised lexicon oriented techniques are applicable in the semantic orientation of public opinions about numerous real world entities. It is observed that socio channels contain real-time contents, which sometimes face the intricacy of informality, Slangs, Vernacular (Native terms), and sarcasm; however, these indicators provide high visibility of sentiments and opinions in terms of orientation. Unfortunately, the unclear perceptiveness of such contents lack in optimized orientation, and supervised machine learning systems are inappropriate where the Lexicon based opinion mining methods are preferred over learning based ones when training data is not adequate. In this paper, we seek to improve the performance of lexicon-based sentiment analysis by incorporating novel linguistic features such as vernaculars, slangs, and sarcasm for monitoring the social media contents up to a more realistic level. The core contributions are sarcasm detection and identification of vernacular terms. The performance of the proposed unsupervised lexicon-based framework over native, informal, and sarcastic opinion bearing terms is assessed via numerous experiments. For this, we utilized tweets relevant to two key domains, including Product and Politics. Experimental outcomes revealed that the proposed system outperformed the existing supervised and semi-supervised systems as 84.24%, and 82.35% of accuracies are achieved over informal and sarcastic contents for product and politics domains, respectively. The average accuracy for both domains is 83.29%.


Introduction
*The rapid advent of social networking sites comes with newer communication channels such as Twitter, Tumblr, and Facebook, etc. Twitter is gaining high popularity among individuals and organizations. Microblogging sites are designed for fast and real time communication. Especially, Twitter is most prominent among the others. Twitter allows a maximum of 140 characters sentence for communication.
Today, users from diverse geographic locations are moving towards microblogging sites in order to share their views, suggestions, speculations due to the wide range coverage and fast communication of these sites. Therefore billions of user-generated content are available about many real world entities. In fact, it is proved as a valuable source of communication for politicians, actors, sportsmen, and even religious scholars. There exist huge availability of public opinions and suggestions in the form of the text, so these contents are helpful in the analysis and prediction of various entities. Social activists, observers, and analysts are curious about knowing, scanning, predicting, and analyzing their desired domains. Extraction and summarization of such valuable opinionative contents are difficult due to variation of expression.
Opinion Mining (Sentiment Analysis) is the only solution that is used to extract public sentiments shared in the form of text. Sentiment Analysis (SA) is a digital recognition of users' opinions through text. It is the process of extracting user-generated content and classifying into different subjective classes.
SA is the problem of Natural Language Processing and the field of text mining. This field has gained great attention due to the proliferation of social networking websites, and hundreds of domains are analyzed through supervised, unsupervised, and lexicon-based methods. SA and OM are now become the auspicious field of research due to the high visibility of application areas such as politics, products, commerce, tourism, hotels, health, services, and education. Also, it is observed that politics (Pontes et al., 2018) and product (Chumwatana, 2018) domains have also gained marvelous attention in comparison with other areas. Although various challenges of socio communication have been solved, there exist few issues which need proper attention from linguistic engineers, i.e., Context-based Aspect level mining, Spam detection, sarcasm detection (Bouazizi and Ohtsuki, 2016), and text informality (Arif et al., 2017) are the core challenges of modern linguistic, and many noticeable experiments have been employed to capture the informality and sarcasm in text for effective sentiment analysis (Bilal et al., 2016;Lo et al., 2017). Sentiment Analysis uses machine learning algorithms, natural language processing tasks, and artificial intelligence mechanisms to cope with the task of classification and analysis. Generally, there exist three main approaches of sentiment analysis; Supervised, unsupervised, and Semi-Supervised. Supervised algorithms need training and testing of data, whereas unsupervised one doesn't need any training and testing; instead, it uses lexicons and dictionaries. On the other hand, semi-supervised uses both pieces of training, testing as well as dictionaries and lexicons. Although hundreds of real world entities have been examined via numerous supervised and unsupervised methods, there is a lack of efficiency in the performance of existing systems. This is due to the informal nature of social media text, as online publisher posts slangs, acronyms, emoticon, and other NetLingua during the conversation, which is comparatively complex to handle. In this way, this research aims to develop a system that can handle informal text in sentiment analysis. Furthermore, it is observed that few users belonging from diverse domains express opposite polarity terms or intensified positive and negative terms in order to present their anger, sadness, and madness. In linguistic such expression can be termed as ironic and sarcastic expression. A sarcastic statement is mostly based on the past context, so it becomes difficult not only to the machine but also to the individual who is unaware of the context. Sarcasm detection is a new research task to SA in the era of modern linguistic. In Asian countries, politicians and politics are treated as a topic of gossip and humor, so the followers of one party usually adopt a sarcastic tone in sharing opinions about the opposing party and vice versa. Analysis of such ambiguous and reverse meaning contents is a challenging task. Therefore, an additional aim of this work is to handle ironic/sarcastic opinions along with informal and native language terms in order to improve the efficiency of the sentiment analysis system. We have developed an unsupervised framework for the semantic orientation of formal, informal, native, and sarcastic opinions over a dataset of products and politics. The source data is collected from Twitter using Twitter APIs and normalized to perform comprehensive experiments in order to evaluate the robustness of the proposed framework. The experimental setup elaborates that the proposed system outperformed the existing systems by achieving 84.24% and 82.35% of accuracies over product and politics domains. We must encourage researchers to participate in SA and OM actively. The rest of the article is comprised of section 2 presents Literature Review, section 3 shows Methodology, section 4 presents Results and Discussion, Section 5 presents Comparative Analysis, and Section 6 presents Conclusion and Future Work.

Literature review
The proliferation of social media sites has produced billions of valuable content that convey useful knowledge about various real world entities. In fact, recent advancement in the web has made it open word of mouth as every real-world entity is discussed. Informative content is increasing day by day with the rapid growth of these social networking sites. Especially the progression of Twitter and Facebook has changed the general trends as people use these sites for sharing their views about products, movies, health education, services, events, individuals, and politics (Hasan et al., 2018). These feelings and sentiment are mined and evaluated for better decision making. Stephen (2010) reviewed the impact of web 2.0 technologies; he stated that social media had replaced the traditional style of observations, surveys, and even interviews. Lai (2010) stated that traditional methods of investigating polls and surveys are time-consuming and labor-intensive tasks, so Twitter and other microblogging services are the best substitutes of these traditional methods, as Twitter allows its user to share contents about any real-life entity. Initially, there was a lack of resources as one can't collect and summarize all desired informative contents, but the advent of sentiment analysis is proved as exultant news for researchers, surveyors, and analysts (Pang and Lee, 2008;Liu, 2015;Zhao et al., 2016). Sentiment Analysis is the study of public attitudes, views, feelings, and sentiments. It is a computational study of public opinions shared in the form of text. The term sentiment analysis was first used by Nasukawa and Yi (2003) in which they performed binary classification on public opinions shared in the form of text while, on the other hand, a very similar term "Opinion Mining" was first seen in Dave et al. (2003). Although Sentiment Analysis and Opinion Mining are two different terms, both terms present the same concept of detecting and scanning public behaviors about desired entities. It is actually the multi-disciplinary area of Natural Language Processing (NLP), Machine Learning (ML), Statistics, and Data Mining, so one can't achieve the desired outcomes without following the necessary linguistic rules of all these disciplines. In fact, machine learning and natural language processing act as a backbone in mining public opinions efficiently (Raza et al., 2017). There exist a lot of work in Sentiment analysis and opinion mining, and hundreds of research problems are uncovered effectively in this specific area (Osimo and Mureddu, 2012;Liu and Zhang, 2012;Yue et al., 2019;Zhang and Liu, 2016;Chaudhuri, 2019). The subsequent section explores the existing research and experiments performed in the desired context.
Social networking websites produced novel communication styles that infer fewer challenges in mining and analysis of public opinions such as slangs, acronyms, and informal sentiment words. In the last few years, much work has been done for informal and non-standard text classification, but still, there is no proper mechanism for efficient classification of holistic non-standard and informal opinions (Mehmood et al., 2019). Furthermore, the context of opinion is not handled properly due to the avoidance of complex but meaningful linguistic clues. Here efficient classification refers to the mean of classifying opinions according to conveyor's sense. The aim is to identify the true positive and negative rate because most reviews over social networking websites are ironic in nature, which badly affect the sentiment analysis accuracy. It is a fact that informal indicators and sarcastic clues act as a backbone in efficient sentiment analysis, so in this section, we have highlighted the approaches, limitations, and inadequacies of past research about formal, informal, and sarcasm detection in opinionative text classification. Kiritchenko et al. (2014) performed sentiment analysis tasks over social media sites for informal text. They used statistical text classification methods for SemEval SMS Datasets and Corpus of movie review excerpts. Their contribution was the automatic generation of lexicons, but their approach was limited and needed highly optimized statistical constructs. Thelwall and Wilkinson (2010) proposed a machine learning algorithm for detecting the sentiment strength of informal terms from MySpace comments, and the strategy adopted was not much effective for informal terms and also requires improved linguistic processing. Amiri and Chua (2012) proposed an optimized approach for the construction of opinion lexicons from cQA services. They used a semi-supervised approach for the detection of Urban and slang terms. The problem with this approach was the utilization of complex graph-based connections for efficient polarity classification. Rout et al. (2018) proposed a model for the detection of sentiments and emotions from informal and unstructured social media contents. They employed a hybrid approach by using supervised and unsupervised algorithms on various datasets. They extracted tweets from Twitter for automatic identification of emotions through an unsupervised approach, and machine learning algorithms such as ME, SVM, and MNB are used with different combinations of features for efficient sentiment analysis. They achieved 80.68% accuracy using the unsupervised approach and 67% accuracy through MNB Algorithm by using bigram and POS features for efficient classification of sentiments from unstructured text. Hamdan (2016) examined the impact of features on sentiment analysis accuracy and experimented supervised machine learning system for sentiment analysis to contribute in SemEval-2016, but the sentiment context is not handled properly and needs automatic construction of lexicons. Balahur (2013) performed sentiment classification tasks over Twitter data and considered tweet structure, length, and language style for the detection of slang and informal terms by using a supervised machine learning approach over the SemEval-2013 dataset. This study suggested five key points that enhanced the classification accuracy, which is; optimized preprocessing (Slang and multilingual definition), Minimal Linguistic processing (Portability), Including the higher level of n-grams (Efficient Classification), use of heuristic (Efficient utilization of features) and applying SVM (for simple/data realistic). Thelwall (2017) proposed a classifier, TensiStrength, for the detection of expression of stress and relaxation with informal social media text. He experimented with two versions of Tensistrength classifiers, along with the utilization of dictionaries, lexicons, and corpus. He used the dictionary of stress and relaxation terms in comparison with the Generic Machine learning method. He concluded that GML performs better in stress and relaxation detection than of Tensistrength. Yang et al. (2013) proposed a supervised SVM based classifier for the classification of short text. They combined lexicon and semantic features and experimented with two key datasets; Google snippet and Ohsumed. They considered five domains from each extracted dataset, while Wikipedia is utilized as background knowledge. They concluded that their method is stable for less number of topics as they achieved 93.87% accuracy with 60 topics. Pennell and Liu (2014) proposed a noisy channel approach for normalizing the informal text. They experimented with two-character leveltechniques. The first one is statistical, and the second one is the Machine translation (MT) model. In comparison, they concluded that their model outperformed the other existing methods (Choudhury et al., 2007;Cook and Stevenson, 2009). Their MT model was based on abbreviation and language modeling. Table 1 shows techniques and addresses problems in sentiment analysis of an informal text. Balahur et al. (2014) surveyed and analyzed the recent trends and computational approaches in addressing the problems of subjectivity and sentiment analysis. She concluded that there exists a sufficient gap in three directions; existing classification dimensions need to be improved for newer challenges, handling informal style of communication. Balahur et al. (2014) also suggested that which approach is beneficial for adaptation from academic to industrial applications.  (2013) Informal Supervised approach Sentiment classification of informal contents shared for electronic products product over social media Mullen and Malouf (2006) Informal Combined Model Investigation into Sentiment Analysis of Informal Political Discourse Mullen and Malouf (2006) investigated political discourse by considering major tasks of NLP, such as shallow parsing and co-reference resolution. Their results suggested that word-based text classification methods are inadequate, while the combination of models generates efficient results in the domain of politics. Balahur and Jacquet (2015) reviewed the sentiment analysis research via a workshop and suggested fewer directions. They highlighted that sentiment analysis could easily meet social media contents if NLP applications and some other features of social sites unfolded clearly. The features they highlighted are; user profile, multilingualism, links of a social media user profile, geolocation, type of message detection, and publishing trends of authors. Lo et al. (2017) Surveyed sentiment classification experiments for multilingual formal and informal text. They reviewed recent research in the field of Sentiment analysis from the year 2004 to 2016 in which ninety-plus articles are cited. They concluded that ignoring informal opinion badly affect the outcomes of sentiment analysis. They stated it is not possible to analyze the desired entity accurately without considering informal and multilingual text. Bilal et al. (2016) performed sentiment classification tasks on Roman-Urdu text extracted from blogs. They used sentence-level classification with three core algorithms; Naïve Bayes, Decision Tree, and KNN. They trained these three classifiers with 300 opinion bearing terms of 150 positive and 150 negative. They concluded that Naïve Bayes outperformed the other two classifiers in sentiment classification. Mataoui et al. (2016) proposed a lexicon-based approach for Algerian Arabic sentiments. They classified the public sentiments shared in the Algerian Arabic language from Facebook. Their proposed system is comprised of four modules; similarity computation, preprocessing, language detection, and polarity classification. They achieved 79.13% accuracy. Dias and Roy (2016) performed a language recognition task for short and informal text. A general-purpose model is used in the identification of short text and transliterated Arabic and Russian words. They stated that language identification could be improved if we incorporate microblogging features. Khan et al. (2016) proposed an optimized framework for sentiment classification of informal opinions. They actually utilized three key features, Bag of words, SWN, and utilization of emotion icons. They enhanced the preprocessing phase by incorporating an informal opinion definition phase. They build three heterogeneous unsupervised classification techniques based on each selected feature, i.e., emoticon classifier, Bag of words classifier, and SWN classifier. They evaluated the performance of their framework over six different datasets and achieved about 85% of average accuracy. Vilares et al. (2017) addressed the rising problems of multilingual sentiment analysis and proposed a unique approach in which they handled the limitations of the existing system. Their state of art supervised classification method utilized the syntax-based rules of unsupervised algorithms by incorporating the feature of interpretability and robustness. They applied the proposed system over different datasets to ensure the domain independency adaptation of the framework and concluded that this system proved as a good source for domain-independent and multilingual sentiment classification as it achieves 74% of average accuracy at different datasets. Neethu and Rajasree (2013) performed sentiment analysis for an electronic product as Mobile and Laptops using machine learning approaches for informal opinionative content. They concluded that in domain-specific sentiment analysis, it is possible to find the impact and effect of sentiment information. They used SVM, NB, ME, and ensemble classifiers, and results demonstrate that all these classifiers produced effective outcomes. Basiri et al. (2014) addressed the problem of Persian language sentiment analysis and offered an unsupervised framework. They developed a very first Persian language opinion lexicon in order to detect the misspelled, informal and nonstandard term in sentiment classification. Their results show that the lexicon-based approach outperformed the machine learning classifier over online cell phone reviews data sets for the Persian language.
In this section, we have thrown light on existing systems, techniques, methods, and approaches used for informal and non-standard text classification in sentiment analyses. Existing research explores that there exist many supervised and unsupervised strategies that are experimented for informal text classification but still, these methods are not fully capable of handling informal and non-standard opinion bearing terms. Few experiments (Kiritchenko et al. 2014;Thelwall and Wilkinson, 2010;Amiri and Chua, 2012;Balahur, 2013) discovered that supervised machine learning systems are inappropriate for informal and nonstandard text because supervised systems need linguistic features and proper training of informal opinion bearing terms. Similarly, Sarcasm is also one such issue that disturbs the sentiment orientation. It is obvious that Sarcasm itself is not an opinion indicator, but it has a great influence on opinion bearing clues. Actually, Sarcasm is an expression or sentence used to taunt or to convey contempt feelings towards the listener. The presence of such linguistic clues in tweets distressed the semantic determination due to misinterpretation of sense. In fact, this linguistic mark has direct impacts on the precision, recall, and other parameters of classification. In the past, Van Hee et al. (2018) participated in SemEval-2018 task3 for ironic tones detection in English text. In order to perform classification, a dataset of sarcastic and ironic tokens is extracted, and also training corpus of 3,834 tweets and a test corpus of 784 tweets is employed to mark the outcomes of their system. They concluded that the proposed system achieved 71% F-Measure on binary, whereas 51% of F-Measure is retrieved on multiclass fine-grained sentiment classification. Kumar et al. (2019) proposed soft attention based Bidirectional Long Short Term Memory (sAtt-BLSTM) Model for the consideration of sarcastic and ironic text in which a convolution neural network is used for assessing the contemporary aspects of sarcasm on balanced and random tweet datasets. They performed experimental studies to monitor the performance of their proposed Deep Learning-based sAtt-BLSTM Model with existing systems. Their findings presented that the proposed sAtt-BLSTM model outperformed others on superior sarcasm classification with 97.8% of accuracy over balanced Twitter data, whereas 93.7% accuracy is attained over a random set of tweets. Riloff et al. (2013) experimented on the sarcastic and ironic situation through a bootstrapping algorithm that automatically identifies the sarcastic situations in the fine-grained orientation of public sentiments. Bamman and Smith (2015) classified the sarcastic moods into positive and negative by identifying tremendous linguistic rules through a supervised machine learning approach. The rules they utilized are; Author's Properties, Audience Properties, and Communicative environments. They achieved 80% of average accuracy on the dataset previously used by González-Ibánez et al. (2011) with features tweet, author, audience, and response. Joshi et al. (2017) explored that sarcasm may be detected by utilizing one of three trends; Semi-Automatic pattern identification for implicit opinions, utilization of subject/topic sign (hashtag) based supervision, and utilization of contextual meaning of posted contents. Dalmia et al. (2015) participated in SemEval 2015 task-10 for three clauses labeling of opinion in message-level task with SVM based supervised machine learning approach. They performed classification tasks on sarcastic text with various feature combinations and concluded that the most appropriate features are those who utilize prior polarities. Similarly, Liebrecht et al. (2013) developed a system for detecting sarcasm from Twitter data in sentiment analysis. They trained the classifier over 78 thousand tweets in which the hashtag symbol is used as a feature for sarcasm detection. Fersini et al. (2015) proposed a Bayesian Model Averaging system for detecting sarcastic tones in microblogging text. Reyes et al. (2013) proposed a multidimensional approach for the recognition of ironic and sarcastic opinions using textual, linguistic features from micro text communication. Although exploring sarcasm through sarcastic words is a common phenomenon, but it is observed that numerical tokens may also contain implicit sarcasm in it for Example, "Pilot: I love to wake up 3 o'clock in the morning" Here in this tweet, time is the only factor which marks it as sarcastic because nobody can love to wake at 3 AM but detecting sarcasm through numeric value is somehow very much difficult to identify. Recently Dubey et al. (2019) examined the numerical part of the text in the identification of sarcasm through deep learningbased statistical machine learning classifier and reached 93% F-Measure on numerical value based Sarcastic Dataset. Further, they are planning to propose a language model for handling of invisible situations in order to improve sarcasm classification performance. Due to this evidence, sarcasm needs to be tackled resourcefully. In this way, the recommended study has highlighted the existing sarcasm detection methods and added a phase of effective sarcasm detection in the proposed framework.

Methodology
To cope with the problem of informal and sarcastic opinion classification, we have proposed an unsupervised lexicon-based framework. Although numerous machine learning systems are available for formal and informal text, they lack an effective classification of sarcastic and roman language terms due to the scarcity of resources. This study proposes an efficient framework for the detection and orientation of formal, informal, vernacular, and sarcastic tags. The major aim of this system is to provide ease to the community who is active in monitoring social media in order to improve their products, organizations, policies, and even trends. Fig. 1 depicts that our framework is comprised of six essential phases; Data Extraction, Normalization, Subjectivity Classification, Scoring, Sarcasm Detection, and Polarity Assignment. Every single step has its own working procedure. This section presents a detailed discussion of the proposed methodology for the efficient monitoring of social media content.

Extraction
Datasets for experimentation purpose is extracted from microblogging services through Twitter APIs. Two key domains, Product and Politics, are considered to evaluate the effectiveness of the proposed framework, as shown in Fig. 1. Extracted text for each mentioned domain is saved in a separate file for further processing.

Text normalization
It is obvious that extracted text can't directly be used for experimentation as it contains a number of irrelevant and undesired tags that have no role in analysis; instead, it overloads the classification system as well as processing time, which leads to generating inappropriate output. Therefore, text normalization (Preprocessing) is performed to produce quality input in order to improve the overall classification output and accuracy. Text preprocessing is performed by adopting the mechanism of Javed and Kamal (2018) preprocessor in the following incremental manners; Noise Reduction, Parts of Speech Tagging, Stop Word Removal, Stemming, and Lemmatization.

Noise reduction
Noise reduction is responsible for removing noise from the text. Noise reduction involves the removal of undesired and unnecessary symbols and tags such as URLs, retweet symbols, and special characters. Python toolkit is used in the removal of unwanted text.

PoS tagging
Noise-free text is then passed to parts of the speech tagging phase in order to assign relevant/appropriate tags to each token in the extracted sentence or tweet. Part of speech is a grammatical category that shows the nature of a word, either it is a verb, adverb, adjective, or noun. In linguistic PoS, Tagging is used to explore the nature of tweets. In fact, grammatical tags suggest either a word/phrase is opinionative or not. Python and WordNet lexicon is used to assign the associated part of the speech tag to each extracted token.

Stop word removal
It is obvious that all words in a sentence may not convey opinions or sentiments. In fact, most of them have no role in any phase of classification. Especially the terms which occur most frequently in order to construct a sentence. Stop words are the most frequent words which have no importance in the sentiment classification process, so all such words need to be eliminated before going towards further phases of classification. Python toolkit is used for the removal of stop words.

Stemming and lemmatization
Previously preprocessed text is then passed to Stemmer and Lemmatizer, as stemming aims to convert the inflected form of words into their root form, whereas lemmatization is the aim of converting infected forms into their meaningful base form. Lemmatization produces more efficient and meaningful results than stemming due to the incorporation of an additional dictionary. Stemming sometimes lowers the precision due to a high falsepositive rate as it converts inflected form without care of meanings and sense (Bird and Loper, 2004).
Lemmatization and Stemming are performed at the same time in order to convert the maximum number of inflected forms into the base, and root forms, respectively, whereas python natural language toolkit is used in the conversion of inflection.

Subjectivity analysis
It is obvious that sentiment classification tags of either positive or negative can't be labeled as nonopinionative tokens. Therefore, it is unavoidable need to identify that non-opinionative tokens before going to the sentiment classification phase. Classification of opinionative and non-opinionative text is termed as subjectivity classification. The proposed framework handled subjectivity in unsupervised manners as subjective lexicons are used to ensure the presence of opinionative tokens. The text, which contains one or more opinionative token, is considered as subjective text; otherwise, it is treated as objective. In addition to existing unsupervised subjectivity classification, the proposed study aims to include informal and vernacular subjective clues for making sentiment analysis more efficient. The clues or features considered in subjectivity classification are adjectives, verbs, Adverbs, slangs, emoticons, and vernacular and sarcastic terms. Python, along with subjective lexicon, is used for the identification and classification of opinionative tweets.

Scoring module
Score Assignment to each target token is the key phase of the proposed framework. Opinion bearing text is the input, whereas the scoring module is a three-step process, namely, scoring of standards indicators, non-standards indicators, and sarcastic opinion indicators. An unsupervised lexicons and dictionaries based score assignment strategy is adopted to cope with these three different types of opinionative indicators.
Algorithm 1 in Table 2 shows the scoring mechanism of both formal and informal opinion indicators, in which the formal opinionative tokens are adjectives handled in Algorithm 1 Line 15-17 and verbs handled in Algorithm 1 Line 18-20, whereas informal includes slang handled in Algorithm 1 Line 21-23 and vernacular handled in Algorithm 1, Line 24-26. Furthermore, nonverbal emotion expressions are also handled in terms of emoticons in the Algorithm 1, Line 12-14, and intensifiers are handled in Algorithm 2 in order to monitor the social media effectively through public sentiments shared in the form of text about product and politics.
Whereas on the other side, the resources mentioned in the proposed model and Algorithm 1 are the sentiment lexicons and dictionaries. Here Sarcastic tags, negators list, and vernacular lexicons are created manually; however, existing sentiment lexicon and slang dictionaries such as Sentiwordnet (SWN) and Noslang are utilized for capturing both formal and informal opinion bearing terms. Then each tweet is passed to the next phase for negation handling and sarcasm detection.

Sarcasm detection
Sarcasm is one of the novel opinion indicators that appeared in microblogging contents. It is difficult to interpret the sense of text for a machine and even to a human on roads of sarcasm detection. Similarly, negators are also the crucial polarity shifters in sentiments from positive to negative and vice versa. An algorithmic mechanism, as shown in  Table 3, presented the detection of sarcasm and negation. Each formal and informal opinion indicator is scanned and verified through negation and sarcasm lexicon for efficient detection of sentiments.

Scoring of intensification features
It is also observed that tweets may contain intensified tones such as exclamation marks, repetition of characters, and use of caps in opinion expression. These opinion bearing clues emphasize the strength of sentiments; therefore, such opinion clues can be referred to as intensifiers. This study captured three major opinion intensifiers to support the sentiment polarity up to a more real and contextual level. Intensifiers in itself are not the prominent indicator, but they act as multiplicative factors in efficient sentiment classification because such clues multiply/enhance the normal weightage of opinion. E.g., Super has a high weight than super. In Table 4, Algorithm 2 shows the scoring of intensification features in which log is used to compute the sentiment weightage of these opinionative clues due to the fact that these features are just additional factors in sentiment classification.

Polarity assignment
The last phase of the proposed framework is the assignment of polarity tags to each analyzed tweet according to its score. The tweet extracted from the sarcasm detection phase is checked. If its score is greater than zero, it will be labeled as positive. Otherwise, it will be marked as negative. At the end sum of the weight of emoticon, adjective, verb, adverb, slang, and vernacular is the tweet sentiment weight, as shown below in Eq. 1.   , never, not, none, nothing, nor, neither, nobody, nowhere, isn't, can't, cannot, don't, didn't, doesn't, hasn't, haven't, hadn't, needn't, mustn't, mightn't, wouldn't, shouldn't, shan't.} Similarly, intensification weight is computed through Algorithm 2. Cumulative weight is between -1 and 1, which is achieved through the sum of weights having verbs, adverbs, adjectives, slangs, sarcasm, vernaculars, and intensification over the number of opinionative tokens as shown below in Eq. 2.
The positive score suggests that the author's opinion and intentions about the concerned entity are positive, and similarly, for the negative score it is marked as negative. Table 5 shows sentiment orientation and polarity tags assignment via the proposed framework.
(1) As discussed earlier, socio communication may contain numeral opinion indicators such as formal (adjective, verb, and adverb), informal (slang, vernacular, and emoticon), and sarcastic tokens. Table 5 illustrates the variance of sentiments shared in the form of text. It is key attainment of the proposed system that it captured the maximum of tweets up to a more real level of sense, but few of the tweets are mishandled due to context, implicit sentiments, and misspelled opinionative tokens.

Results and discussion
This section presents the results achieved through the experimental process of the proposed framework. An equally distributed set of data is assessed deliberately in order to unfold the insights of the proposed framework by evaluating precision, recall, F-measure, and accuracy over a dataset of 13000 tweets about two key domains: Product and politics. Table 6 presents the Statistics of Tweets classification.  Table 6 illustrates that a total of 6500 tweets for each domain is investigated for all three mentioned classes; Formal, Non-standard, and Sarcastic. The experimental process revealed that informal sarcastic opinions still need improvement as we achieved 83.29% of average accuracy with a feature set of formal, non-standard, and sarcastic tweets for both domains. Whereas the confusion matrix, along with the evaluation parameters as Precision, recall, fmeasure, and accuracy, is used to present the achieved outcomes in a more clear fashion. The confusion matrix for both domains is shown in Table  7.

Precision
In data mining and information retrieval, precision is the ratio of accessed token that is actually relevant to the desired format.
Mathematically precision can be represented as in Eq. 3; =84.53%

Recall
In data mining and information retrieval, recall is the ratio of relevant tokens that are accessed

F-Measure
In statistical analysis, F-Measure estimates the test accuracy by utilization of both precision and recall. F-measure a. k. a. F1 Score) can be represented mathematically as in Eq. 5;

Accuracy
In statistical analysis, accuracy can be defined as the quality or correctness of target data or instances in terms of true value. Mathematically accuracy can be defined as in Eq. 6; Accuracy=   Similarly, Fig. 3 shows that the incorporation of informal and sarcastic tones has also improved the precision-recall and F-Measure over both positive and negative instances of the Politics Domain. Fig. 4 shows that after the incorporation of informal and sarcastic clues, the accuracies are raised very high to 84.24% and 82.35% with 5.04% and 4.41% of increment on product and politics datasets, respectively. It is observed that consideration of these three formal, informal, and sarcastic clue have strengthened the sentiment analysis system by improving the accuracy, precision, recall, and F-Measure on both domains. Fig. 4 shows the clear differences in outcomes in the view of with and without consideration of these three linguistic clues.

Comparative analysis
Keeping in view the target problem, datasets, and framework, the proposed study is analyzed in comparison with existing studies, and it is observed that Socio Monitoring Framework (SMF) outperformed the other supervised and unsupervised sentiment analysis system over formal, informal, and sarcastic contents. Table 8 illustrates that the proposed framework achieved better accuracy than others except Kundi et al. (2014) on formal, informal, and sarcastic tweets of product and politics domains. They achieved better accuracy just due to the fact that they ignored the vernacular and sarcastic clues, whereas the proposed system enhanced the handling of linguistic clues in order to extract the maximum number of opinionative contents, which causes the reduction in accuracy but improvement in the capturing of robust data.

Conclusion and future work
Sentiment Analysis is the up growing research arena that is useful in estimating and assessing the popularity and obscurity of products, policies, and services. Opinion Mining and sentiment analysis help in the analysis of various domains and entities, including Mobile Phones, services, political entities, political policies, and political events. Machine learning act as a backbone in statistical and computational analysis of public reviews; Supervised Learning Algorithms such as SVM (Support Vector Machine), Naïve Bayes, and Maximum Entropy, Similarly Semi-supervised and unsupervised ML algorithms are used in sentiment analysis and opinion mining. To handle informal and sarcastic text, lexicon-based opinion mining techniques are preferred over existing machine learning algorithms.
This study proposed a socio monitoring framework for the detection and orientation of public expressions shared in the form of informal, vernacular, and sarcastic text about two key domains: Product and Politics. The novel contribution of the proposed study is to give identification to Vernaculars, slangs, and sarcasm detection. In order to evaluate the performance of the proposed socio monitoring framework, a couple of experiments are performed on target datasets. Results demonstrate that the proposed framework achieved promising outcomes on comparative analysis of with and without incorporation of informal features on target datasets. Moreover, after the incorporation of sarcastic clues, the accuracies are raised very high to 84.24% and 82.35% with 5.04% and 4.41% of increment on product and politics datasets, respectively. The conclusive observation states that consideration of these three formal, informal and sarcastic clue have strengthened the sentiment analysis system by improving the accuracy, precision, recall, and F-Measure, but there is a need of improvement in informal and sarcasm detection because we handled well known informal text whereas there exist huge variation of slangs and informal terms similarly for sarcasm we just considered the explicit sarcastic tags whereas much of the text contains an implicit and vague form of sarcasm which is sometimes unrecognizable not only to a machine but even to a human. To cope with these two limitations, we are planning to enhance the size of the dictionary for informal sentiment analysis, and similarly, novel strategies for efficient sarcasm detection need to be added in order to enhance the performance of the proposed socio monitoring framework.