Service Discovery Protocols

Service Discovery is the automatic detection of devices and services offered by devices on a computer network. A service discovery protocol (SDP) is a network protocol that helps accomplish service discovery. 

Simple Service Discovery Protocol (SSDP) is a networkprotocol based on the Internet Protocol Suite for advertisement and discovery of network services and presence information. It accomplishes this without assistance of server-based configuration mechanisms, such as DHCP or DNS, and without special static configuration of a network host. SSDP is the basis of the discovery protocol of Universal Plug and Play (UPnP) and is intended for use in residential or small office environments. 

Multicast DNS(mDNS) protocol is published as RFC 6762, uses IP multicast UserDatagram Protocol(UDP) packets, and is implemented by the Apple Bonjour and open source Avahi software packages. Android contains an mDNS implementation. mDNS has also been implemented in Windows 10, but its use there is limited to discovering networked printers. mDNS can work in conjunction with DNS Service Discovery (DNS-SD), a companion zero-configuration technique specified separately in RFC 6763

DNS Service Discovery over Multicast DNS (DNS-SD/mDNS), made fashionable through Apple’s Bonjour. DNS Service Discovery over Multicast DNS (DNS-SD/mDNS) is a prevalent technique widely used for offering and requesting services in local networks without configuration. Using the upper two layers of the Zeroconf stack, namely DNS Service Discovery and Multicast DNS, it provides a great user experience. DNS-SD/mDNS is widely used. It runs on Linux (Avahi), Windows (Avahi, Bonjour), MacOS (Bonjour), Android (NSD), and iOS (Bonjour). Implementations for Internet of Things (IoT) operating systems, such as contiki, also exist.

Zeroconf stack provides configurationless means for all of addressing, name resolution, and service discovery. A huge advantage is that these layers are independent of each other. The name resolution mechanism - multicast DNS (mDNS) works with automatic  or static address configuration as well as with DHCP. The service discovery layer - DNS service discovery (DNS-SD)  works with standard DNS as well as mDNS.

ref:

Service Discovery - https://en.wikipedia.org/wiki/Service_discovery

Protocols for Device Discovery - https://www.bbc.co.uk/rd/blog/2014-07-protocols-for-device-discovery

Service Discovery using ZeroConf Stack(DNS-SD/mDNS) - https://pdfs.semanticscholar.org/0c62/d94cef19690d8f1fabc7e1f8bcf369dc49ce.pdf

SSDP(Simple Service Discovery Protocol) Protocol - https://en.wikipedia.org/wiki/Simple_Service_Discovery_Protocol

MultiCast DNS(mDNS) -

• https://en.wikipedia.org/wiki/Multicast_DNS

• http://www.multicastdns.org/ 

• http://www.zeroconf.org

DNS-SD overview -  http://www.dns-sd.org/. https://en.wikipedia.org/wiki/Zero-configuration_networking#DNS-SD

Apple Bonjour-

• Overview - https://developer.apple.com/bonjour/

• Source code (tar ball) - https://opensource.apple.com/tarballs/mDNSResponder/

• Bonjour Concepts (Documentation) - https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/NetServices/Articles/about.html

SSDP protocol opensource libraries-

• https://github.com/kallisti5/libmicrossdp

• https://github.com/zlargon/lssdp

• https://github.com/troglobit/ssdp-responder

• https://github.com/topics/ssdp

Multicast DNS(mDNS) & DNS-SD Overview - https://meetings.ripe.net/ripe-55/presentations/strotmann-mdns.pdf

Apple mDNS/ZeroConf "C" library: Bonjour (Apache License) - https://opensource.apple.com/source/mDNSResponder/mDNSResponder-214/mDNSCore/

Linux Avahi library source code (LGPL license) - https://github.com/lathiat/avahi

Google chromium mDNS source code - https://chromium.googlesource.com/chromium/src/+/68ea490084597d5d4640e782989c0a6a094dcd21/chrome/browser/extensions/api/mdns

Multicast DNS and DNS-SD for the Spark Core - https://github.com/mrhornsby/spark-core-mdns, https://community.particle.io/t/mdns-and-dns-service-discovery-library/9550/3

Apple Bonjour “C” library Posix Port - https://opensource.apple.com/source/mDNSResponder/mDNSResponder-541/mDNSPosix/

Apple Bonjour Test/Sample code - https://github.com/jevinskie/mDNSResponder

ZeroConf(mDNS) discovery in C++ - https://github.com/HBPVIS/Servus

mDNS opensource implementations - https://github.com/topics/mdns 

Open IOT LIbraries - https://github.com/Agile-IoT/awesome-open-iot 

Multicast DNS(mDNS) & DNS-SD - https://webscreens.github.io/openscreenprotocol/mdns.html

Android Bonjour mDNS responder - https://www.andriydruk.com/post/mdnsresponder/

DNS Service Discovery on Windows - https://marknelson.us/posts/2011/10/25/dns-service-discovery-on-windows.html

Bonjour overview - https://people.eecs.berkeley.edu/~johnw/cs294-97/papers/Bonjour%20-%20Overview.pdf

SIP URI Service Discovery using DNS-SD - https://tools.ietf.org/html/draft-lee-sip-dns-sd-uri-03

Light-weight multicast DNS and DNS-SD (lmDNS-SD) - https://www.researchgate.net/publication/261427432_Light-Weight_Multicast_DNS_and_DNS-SD_lmDNS-SD_IPv6-Based_Resource_and_Service_Discovery_for_the_Web_of_Things

Proxy support for service discovery using mDNS/DNS-SD in low power networks - http://www.win.tue.nl/~mstolikj/publications/IOTSOS2014.pdf

KDNSSD(Network service discovery using Zeroconf) - https://api.kde.org/frameworks/kdnssd/html/index.html

Machine Learning(ML) Overview

Machine learning is a Umbrella term. It is a type of artificial intelligence (AI) that provides computers with the ability to learn without being explicitly programmed. AI means making computers act intelligently. It is one of the major fields of study in computer science and encompasses sub-fields such as robotics, machine learning, expert systems, general intelligence and natural language processing.” Machine learning focuses on the development of computer programs that can teach themselves to grow and change when exposed to new data.

Machine learning is the sub field of computer science that "gives computers the ability to learn without being explicitly programmed" (Arthur Samuel, 1959). Evolved from the study of pattern recognition and computational learning theory in artificial intelligence, machine learning explores the study and construction of algorithms that can learn from and make predictions on data such algorithms overcome following strictly static program instructions by making data-driven predictions or decisions through building a model from sample inputs.

Machine learning is closely related to (and often overlaps with) computational statistics, which also focuses in prediction-making through the use of computers. It has strong ties to mathematical optimization, which delivers methods, theory and application domains to the field. Machine learning is sometimes conflated with Data Mining where the latter subfield focuses more on exploratory data analysis. Statistical Analysis is a component of data analytics. In the context of business intelligence (BI), statistical analysis involves collecting and scrutinizing every data sample in a set of items from which samples can be drawn.

Deep learning is a form of machine learning that uses a model of computing that's very much inspired by the structure of the brain. Hence we call this model a neural network. The basic foundation unit of a neural network is the neuron, which is actually conceptually quite simple.

Machine learning tasks are typically classified into three broad categories, depending on the nature of the learning "signal" or "feedback" available to a learning system. These are -

    1. Supervised learning: The computer is presented with example inputs and their desired outputs, given by a "teacher", and the goal is to learn a general rule that maps inputs to outputs.

    2. Unsupervised learning: No labels are given to the learning algorithm, leaving it on its own to find structure in its input. Unsupervised learning can be a goal in itself (discovering hidden patterns in data) or a means towards an end (feature learning).

    3. Reinforcement learning: A computer program interacts with a dynamic environment in which it must perform a certain goal (such as driving a vehicle), without a teacher explicitly telling it whether it has come close to its goal. Another example is learning to play a game by playing against an opponent.

Generalization refers to how well the concepts learned by a machine learning model apply to specific examples not seen by the model when it was learning. The goal of a good machine learning model is to generalize well from the training data to any data from the problem domain. This allows us to make predictions in the future on data the model has never seen. There is a terminology used in machine learning when we talk about how well a machine learning model learns and generalizes to new data, namely overfitting and underfitting.

Overfitting and underfitting are the two biggest causes for poor performance of machine learning algorithms. Overfitting refers to a model that models the training data too well. Overfitting happens when a model learns the detail and noise in the training data to the extent that it negatively impacts the performance on the model on new data. This means that the noise or random fluctuations in the training data is picked up and learned as concepts by the model. The problem is that these concepts do not apply to new data and negatively impact the models ability to generalize. Underfitting refers to a model that can neither model the training data not generalize to new data. An underfit machine learning model is not a suitable model and will be obvious as it will have poor performance on the training data. Underfitting is often not discussed as it is easy to detect given a good performance metric. The remedy is to move on and try alternate machine learning algorithms.

Supervised Learning:

Supervised learning is the machine learning task of inferring a function from labeled training data. The training data consist of a set of training examples. In supervised learning, each example is a pair consisting of an input object (typically a vector) and a desired output value (also called the supervisory signal). The majority of practical machine learning uses supervised learning. Supervised learning is where you have input variables (x) and an output variable (Y) and you use an algorithm to learn the mapping function from the input to the output.

Y = f(X)

The goal is to approximate the mapping function so well that when you have new input data (x) that you can predict the output variables (Y) for that data.

Supervised learning mainly has 2 categories:

    1. Classification - Target variable is categorical(yes/no). A classification problem is when the output variable is a category, such as “red” or “blue” or “disease” and “no disease”.
    2. Regression - Target variable is continuous. A regression problem is when the output variable is a real value, such as “dollars” or “weight”.

Some popular examples of supervised machine learning algorithms are:
    1. Linear regression for regression problems.
    2. Random forest for classification and regression problems.
    3. Support vector machines for classification problems.

Unsupervised Learning:

Unsupervised learning is the machine learning task of inferring a function to describe hidden structure from unlabeled data. Since the examples given to the learner are unlabeled, there is no error or reward signal to evaluate a potential solution - this distinguishes unsupervised learning from supervised learning and reinforcement learning. Unsupervised learning is closely related to the problem of density estimation in statistics. However, unsupervised learning also encompasses many other techniques that seek to summarize and explain key features of the data.

Unsupervised learning problems can be further grouped into clustering and association problems -

    1. Clustering: A clustering problem is where you want to discover the inherent groupings in the data, such as grouping customers by purchasing behavior.
    2. Association:  An association rule learning problem is where you want to discover rules that describe large portions of your data, such as people that buy X also tend to buy Y.

Some popular examples of unsupervised learning algorithms are:
    1. K-means for clustering problems.
    2. Apriori algorithm for association rule learning problems.

ref:

Wiki -
    1. Artificial Intelligence - https://en.wikipedia.org/wiki/Artificial_intelligence
    2. Machine Learning - https://en.wikipedia.org/wiki/Machine_learning
    3. Unsupervised Learning - https://en.wikipedia.org/wiki/Unsupervised_learning
    4. Supervised Learning - https://en.wikipedia.org/wiki/Supervised_learning
    5. Neural Networks - https://en.wikipedia.org/wiki/Artificial_neural_network

Deep Neural Networks - https://www.technologyreview.com/s/602344/the-extraordinary-link-between-deep-neural-networks-and-the-nature-of-the-universe/

Supervised and Unsupervised learning - http://machinelearningmastery.com/supervised-and-unsupervised-machine-learning-algorithms/

Machine Learning Algorithms - http://machinelearningmastery.com/a-tour-of-machine-learning-algorithms/

Machine Learning using Python - http://scikit-learn.org/

Misc -
    1. http://www.kdnuggets.com/2015/01/deep-learning-explanation-what-how-why.html
    2. http://machinelearningmastery.com/overfitting-and-underfitting-with-machine-learning-algorithms/
    3. http://math.stackexchange.com/questions/141381/regression-vs-classification

Porting Linux Applications to VxWorks RTOS

Tips for Porting Linux Application to VxWorks: 

1. There is no notion of a process (in the POSIX sense), there is no notion of sharing of locks (mutexes) and condition variables between processes.  As a result, the POSIX symbol _POSIX_THREAD_PROCESS_SHARED is not defined in this implementation, and the routines pthread_condattr_getpshared( ), pthread_condattr_setpshared( ), pthread_mutexattr_getpshared( ) are not implemented.

2. There are NO processes in VxWorks, fork( ), wait( ), and pthread_atfork( ) are unimplemented. If you need to spawn a task for that using the 'taskSpawn' routine.

3. VxWorks does NOT have password, user, or group databases, therefore there are no implementations of getlogin( ), getgrgid( ), getpwnam( ), getpwuid( ), getlogin_r( ), getgrgid_r( ), getpwnam_r( ), and getpwuid_r( )

4. VxWorks programming discourages to use global variables and if in case you need to use the global variables ( global scope ,file scope ) make sure you initialize those before your application program starts and make sure use to use the unique variable names.This is because of the fact that vxWorks memory model is flat,so you need to be extra careful in using the global variables.

5. The 'main' routine does not work in vxWorks case, We need to write your own wrapper function to parse the command line arguments that you pass to your program from the interactive shell.

6. It is recommended to use the separate memory partition for your application if your application demands more frequent use of dynamic memory allocations and frees.

7. It is better to use the static memory for small memory allocations because vxWorks mem library does not use best fit  algorithm for its memory allocations and it can lead to memory fragmentation.

ref:

http://www.vxdev.com/docs/vx55man/vxworks/ref/pthreadLib.html

https://groups.google.com/forum/#!topic/comp.os.vxworks/PVGKsSaGxtw

Porting RTOS drivers to Linux - http://www.linuxjournal.com/article/7355

VxWorks API to Linux - http://v2lin.sourceforge.net/

Migrating legacy VxWorks applications to Linux - http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.466.7007&rep=rep1&type=pdf