[OSC_dev] State of OSC 1.1

[Kaspar Bumke]

Hi all,

Just been wondering about the state and plans for OSC 1.1. The paper
outlining it is from 2009 and in the description of additional types for
instance it says "These may be found in the specification and are omitted
here in the interests of brevity". But there seems to be no page with the
full specification. I understand the finalization has been put on hold due
to budget cuts but is there any sort of plan for it?

I am especially interested in the SLIP USB-Serial specification and would
find it really great if it would actually be started to be used by
soft-synth and DAW developers. This would allow for a whole range of new
OSC devices that are USB only.

Regards,

Kaspar

[Andy W. Schmeder]

Just been wondering about the state and plans for OSC 1.1. The paper outlining it is from 2009 and in the description of additional types for instance it says "These may be found in the specification and are omitted here in the interests of brevity". But there seems to be no page with the full specification. I understand the finalization has been put on hold due to budget cuts but is there any sort of plan for it?
I am especially interested in the SLIP USB-Serial specification and would find it really great if it would actually be started to be used by soft-synth and DAW developers. This would allow for a whole range of new OSC devices that are USB only.

The plan for OSC 1.1 was never more than to patch up some of the most glaring problems.

I think that to have OSC devices working well with a broad range of applications and connectivity methods requires a lot more effort and probably some improvements to the core format, in particular better type support and the functional "glue layers", i.e., discovery, enumeration and control. Possibly the main issue is implementing and testing all that which is considerable...

About a year ago I started drafting an OSC 2.0 document, similar format to 1.0 but not 100% backwards compatible, ABNF grammar-based, and having a type system and type meta-data descriptors based on ISO/IEC 11404:2007.


The list of types currently outlined for OSC 2.0 is:

Fixed-length primitive data types:

void (N)
bit int 1 (0)
signed bit S1 (1)
int S7 (c)
int S15 (u)
int S23 (a)
int S31 (i)
int S63 (h)
int S127 (j)
unsigned int 8 (C)
unsigned int 16 (U)
unsigned int 24 (A)
unsigned int 32 (I)
unsigned int 64 (H)
unsigned int 128 (J)
fixed-point S7.8 (p)
fixed-point S15.16 (v)
fixed-point S23.24 (w)
fixed-point S31.32 (x)
fixed-point S63.64 (y)
fixed-point S127.128 (z)
unsigned fixed-point 8.8 (P)
unsigned fixed-point 16.16 (V)
unsigned fixed-point 24.24 (W)
unsigned fixed-point 32.32 (X)
unsigned fixed-point 64.64 (Y)
unsigned fixed-point 128.128 (Z)
float 16 (e)
float 32 (f)
float 64 (d)
float 128 (q)


Fixed-length composite type modifiers

dyad (2)
quad (4)
octet (8)
16-tet (E)
32-tet (F)
64-tet (D)
128-tet (Q)
interval pair (-)
complex pair (+)
complex circular interval (*)


Variable length primitive types:

utf-8 string (s)
binary section (b)
dense vector (|)
sparse vector (:)
dense matrix (%)
sparse matrix (@)
unlimited unbounded int n (_)
unlimited unbounded fixed-point n.n (.)


Variable length composite types:

list ([])
object ({})


---

Andy W. Schmeder
email: andy [at] cnmat.berkeley.edu
skype: andy.schmeder

Programmer/Analyst II
Research Group
Center for New Music and Audio Technologies
University of California at Berkeley
http://cnmat.berkeley.edu

[Jeff Koftinoff]

Nice! more info please! When will you release OSC 2.0 ?

Regards,
Jeff

Post by Andy W. Schmeder

Just been wondering about the state and plans for OSC 1.1. The paper outlining it is from 2009 and in the description of additional types for instance it says "These may be found in the specification and are omitted here in the interests of brevity". But there seems to be no page with the full specification. I understand the finalization has been put on hold due to budget cuts but is there any sort of plan for it?
I am especially interested in the SLIP USB-Serial specification and would find it really great if it would actually be started to be used by soft-synth and DAW developers. This would allow for a whole range of new OSC devices that are USB only.

The plan for OSC 1.1 was never more than to patch up some of the most glaring problems.
I think that to have OSC devices working well with a broad range of applications and connectivity methods requires a lot more effort and probably some improvements to the core format, in particular better type support and the functional "glue layers", i.e., discovery, enumeration and control. Possibly the main issue is implementing and testing all that which is considerable...
About a year ago I started drafting an OSC 2.0 document, similar format to 1.0 but not 100% backwards compatible, ABNF grammar-based, and having a type system and type meta-data descriptors based on ISO/IEC 11404:2007.
void (N)
bit int 1 (0)
signed bit S1 (1)
int S7 (c)
int S15 (u)
int S23 (a)
int S31 (i)
int S63 (h)
int S127 (j)
unsigned int 8 (C)
unsigned int 16 (U)
unsigned int 24 (A)
unsigned int 32 (I)
unsigned int 64 (H)
unsigned int 128 (J)
fixed-point S7.8 (p)
fixed-point S15.16 (v)
fixed-point S23.24 (w)
fixed-point S31.32 (x)
fixed-point S63.64 (y)
fixed-point S127.128 (z)
unsigned fixed-point 8.8 (P)
unsigned fixed-point 16.16 (V)
unsigned fixed-point 24.24 (W)
unsigned fixed-point 32.32 (X)
unsigned fixed-point 64.64 (Y)
unsigned fixed-point 128.128 (Z)
float 16 (e)
float 32 (f)
float 64 (d)
float 128 (q)
Fixed-length composite type modifiers
dyad (2)
quad (4)
octet (8)
16-tet (E)
32-tet (F)
64-tet (D)
128-tet (Q)
interval pair (-)
complex pair (+)
complex circular interval (*)
utf-8 string (s)
binary section (b)
dense vector (|)
sparse vector (:)
dense matrix (%)
unlimited unbounded int n (_)
unlimited unbounded fixed-point n.n (.)
list ([])
object ({})
---
Andy W. Schmeder
email: andy [at] cnmat.berkeley.edu
skype: andy.schmeder
Programmer/Analyst II
Research Group
Center for New Music and Audio Technologies
University of California at Berkeley
http://cnmat.berkeley.edu
_______________________________________________
OSC_dev mailing list
http://lists.create.ucsb.edu/mailman/listinfo/osc_dev

[Andy W. Schmeder]

Post by Jeff Koftinoff
Nice! more info please! When will you release OSC 2.0 ?

Good question! :) Obviously it keeps getting shifted onto the back-burner :(

I am going to put on my calendar a draft release for Dec 1 and then we can take a month or two to incorporate everyones complaints before its finalized... sound good?? :)

Post by Jeff Koftinoff

Post by Andy W. Schmeder
About a year ago I started drafting an OSC 2.0 document, similar format to 1.0 but not 100% backwards compatible, ABNF grammar-based, and having a type system and type meta-data descriptors based on ISO/IEC 11404:2007.

---

Andy W. Schmeder
email: andy [at] cnmat.berkeley.edu
skype: andy.schmeder

Programmer/Analyst II
Research Group
Center for New Music and Audio Technologies
University of California at Berkeley
http://cnmat.berkeley.edu

[Andy W. Schmeder]

Yes the "core" types would remain the small set, i,f,s,b, possibly plus the 64-bit h,d. Seems now that most environments are moving towards a 64-bit base for numerics (python, javascript, maxmsp etc).

As far as support for the full set of types, implementations would have to at least be able gracefully handle the condition where an unsupported type is encountered, either by dropping the packet or by skipping over it in the data section, which in itself isn't necessarily trivial unless the protocol makes it easy to compute where the next boundary is...

What's to become of the current distinction between the 3 "standard" types i,f,s and the "optional" types? Looking at your list it seems daunting to implement... Are there envisioned levels of conformance or a subset of types optimized for interchange?
-Matt

---

Andy W. Schmeder
email: andy [at] cnmat.berkeley.edu
skype: andy.schmeder

Programmer/Analyst II
Research Group
Center for New Music and Audio Technologies
University of California at Berkeley
http://cnmat.berkeley.edu

[s***@xs4all.nl]

Post by Andy W. Schmeder

Post by Kaspar Bumke
Just been wondering about the state and plans for OSC 1.1. The paper
outlining it is from 2009 and in the description of additional types for
instance it says "These may be found in the specification and are
omitted here in the interests of brevity". But there seems to be no page
with the full specification. I understand the finalization has been put
on hold due to budget cuts but is there any sort of plan for it?
I am especially interested in the SLIP USB-Serial specification and
would find it really great if it would actually be started to be used by
soft-synth and DAW developers. This would allow for a whole range of new
OSC devices that are USB only.

The plan for OSC 1.1 was never more than to patch up some of the most glaring problems.
I think that to have OSC devices working well with a broad range of
applications and connectivity methods requires a lot more effort and
probably some improvements to the core format, in particular better type
support and the functional "glue layers", i.e., discovery, enumeration and
control. Possibly the main issue is implementing and testing all that
which is considerable...
About a year ago I started drafting an OSC 2.0 document, similar format to
1.0 but not 100% backwards compatible, ABNF grammar-based, and having a
type system and type meta-data descriptors based on ISO/IEC 11404:2007.
void (N)
bit int 1 (0)
signed bit S1 (1)
int S7 (c)
int S15 (u)
int S23 (a)
int S31 (i)
int S63 (h)
int S127 (j)
unsigned int 8 (C)
unsigned int 16 (U)
unsigned int 24 (A)
unsigned int 32 (I)
unsigned int 64 (H)
unsigned int 128 (J)
fixed-point S7.8 (p)
fixed-point S15.16 (v)
fixed-point S23.24 (w)
fixed-point S31.32 (x)
fixed-point S63.64 (y)
fixed-point S127.128 (z)
unsigned fixed-point 8.8 (P)
unsigned fixed-point 16.16 (V)
unsigned fixed-point 24.24 (W)
unsigned fixed-point 32.32 (X)
unsigned fixed-point 64.64 (Y)
unsigned fixed-point 128.128 (Z)
float 16 (e)
float 32 (f)
float 64 (d)
float 128 (q)
Fixed-length composite type modifiers
dyad (2)
quad (4)
octet (8)
16-tet (E)
32-tet (F)
64-tet (D)
128-tet (Q)
interval pair (-)
complex pair (+)
complex circular interval (*)
utf-8 string (s)
binary section (b)
dense vector (|)
sparse vector (:)
dense matrix (%)
unlimited unbounded int n (_)
unlimited unbounded fixed-point n.n (.)
list ([])
object ({})
---
Andy W. Schmeder
email: andy [at] cnmat.berkeley.edu
skype: andy.schmeder
Programmer/Analyst II
Research Group
Center for New Music and Audio Technologies
University of California at Berkeley
http://cnmat.berkeley.edu
_______________________________________________
OSC_dev mailing list
http://lists.create.ucsb.edu/mailman/listinfo/osc_dev

Andy, if I might just say - that looks great from a user point of view,
but having to implement all of those types would be a nightmare from a
developer point of view. Why the need for so many different integer
lengths, and so many different fixed point values ? I would stick to fewer
types as a developer, e.g.

(i) int32 (for backwards compatibility)
(I) int64
(f) float32
(F) float64 (?)
(s) NULL terminated string (for backwards compatibility)
(b) binary data (int64 length followed by length bytes of data)

I would say all we really need are F and b ! Maybe not even F, we could
represent numbers in atol() format.

So a message would be just:


/msg/foo\0<data0_len><data0><data1_len><data1>....<datan_len><datan>\0


Then implement typetags as meta-data, something like:

/msg/foo/.typetags/get


returns a string of typetags.




The advantage of doing it this way:
- client gets to define parameters of message at runtime (discovery)
- client/host can ignore any formats they dont understand (just send 0
length data)
- message format is simplified



Regards,
Salsaman.


Salsaman.
http://lives.sourceforge.net

[Adrian Freed]

Post by s***@xs4all.nl
Andy, if I might just say - that looks great from a user point of view,
but having to implement all of those types would be a nightmare from a
developer point of view. Why the need for so many different integer
lengths, and so many different fixed point values ? I would stick to fewer
types as a developer, e.g.

I am afraid the ship has already sailed on this one.
Even the $5 ARM8 has double, word, half word and byte:
http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0344h/Beidjgca.html
Application libraries (mainly in graphics) have things like half-floats.
I don't think these are particularly challenging for developers but we shall see as we prototype these things...

[Sciss]

the question is if any of those are particularly useful.

i'm more baffled by the fact there was no clear definition as how strings are encoded -- you propose UTF-8 now, which is great. that's a much more relevant decision IMO than adding half a dozen extra primitive types that have so little use in 98% of the applications, that requesting to support them in any implementation is a waste of resources. i think OSC 1.1 was good here with the four types, int32, long64, float32, double64.

best, -sciss-

Post by Adrian Freed

Post by s***@xs4all.nl
Andy, if I might just say - that looks great from a user point of view,
but having to implement all of those types would be a nightmare from a
developer point of view. Why the need for so many different integer
lengths, and so many different fixed point values ? I would stick to fewer
types as a developer, e.g.

I am afraid the ship has already sailed on this one.
http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0344h/Beidjgca.html
Application libraries (mainly in graphics) have things like half-floats.
I don't think these are particularly challenging for developers but we shall see as we prototype these things...
_______________________________________________
OSC_dev mailing list
http://lists.create.ucsb.edu/mailman/listinfo/osc_dev

[Mattijs Kneppers]

Hey sciss,

As long as a developer is free to skip the messages with data types
his program doesn't support, I can't see anything wrong with setting a
standard to support more data types, right?

A more relevant question as I see it is whether actively skipping data
in general (which needs parsing it first, at least to a certain
extent) is something we could try to avoid.

I don't remember whether it is mentioned in the available docs that
sending data to an ip and port should be limited to only the data
relevant for that device. Perhaps a guarantee for that could be looked
into, although I must admit I don't have ideas about how that could be
achieved.

Best,
Mattijs

Post by Sciss
the question is if any of those are particularly useful.
i'm more baffled by the fact there was no clear definition as how strings are encoded -- you propose UTF-8 now, which is great. that's a much more relevant decision IMO than adding half a dozen extra primitive types that have so little use in 98% of the applications, that requesting to support them in any implementation is a waste of resources. i think OSC 1.1 was good here with the four types, int32, long64, float32, double64.
best, -sciss-

Post by Adrian Freed

Post by s***@xs4all.nl
Andy, if I might just say - that looks great from a user point of view,
but having to implement all of those types would be a nightmare from a
developer point of view. Why the need for so many different integer
lengths, and so many different fixed point values ? I would stick to fewer
types as a developer, e.g.

I am afraid the ship has already sailed on this one.
http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0344h/Beidjgca.html
Application libraries (mainly in graphics) have things like half-floats.
I don't think these are particularly challenging for developers but we shall see as we prototype these things...
_______________________________________________
OSC_dev mailing list
http://lists.create.ucsb.edu/mailman/listinfo/osc_dev

_______________________________________________
OSC_dev mailing list
http://lists.create.ucsb.edu/mailman/listinfo/osc_dev

[s***@xs4all.nl]

Post by Mattijs Kneppers
Hey sciss,
As long as a developer is free to skip the messages with data types
his program doesn't support, I can't see anything wrong with setting a
standard to support more data types, right?
A more relevant question as I see it is whether actively skipping data
in general (which needs parsing it first, at least to a certain
extent) is something we could try to avoid.
I don't remember whether it is mentioned in the available docs that
sending data to an ip and port should be limited to only the data
relevant for that device. Perhaps a guarantee for that could be looked
into, although I must admit I don't have ideas about how that could be
achieved.
Best,
Mattijs

Such an approach would be great; but even so,

a) type tags dont have to be hard coded in the message.

b) it would be nice to have some kind of blob type for sending in-channel
data. Either that or store the string length for strings.


Salsaman.

http://lives.sourceforge.net

Post by Mattijs Kneppers

Post by Sciss
the question is if any of those are particularly useful.
i'm more baffled by the fact there was no clear definition as how
strings are encoded -- you propose UTF-8 now, which is great. that's a
much more relevant decision IMO than adding half a dozen extra primitive
types that have so little use in 98% of the applications, that
requesting to support them in any implementation is a waste of
resources. i think OSC 1.1 was good here with the four types, int32,
long64, float32, double64.
best, -sciss-

Post by Adrian Freed

Post by s***@xs4all.nl
Andy, if I might just say - that looks great from a user point of view,
but having to implement all of those types would be a nightmare from a
developer point of view. Why the need for so many different integer
lengths, and so many different fixed point values ? I would stick to fewer
types as a developer, e.g.

I am afraid the ship has already sailed on this one.
http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0344h/Beidjgca.html
Application libraries (mainly in graphics) have things like
half-floats.
I don't think these are particularly challenging for developers but we
shall see as we prototype these things...
_______________________________________________
OSC_dev mailing list
http://lists.create.ucsb.edu/mailman/listinfo/osc_dev

_______________________________________________
OSC_dev mailing list
http://lists.create.ucsb.edu/mailman/listinfo/osc_dev

--
arttech.nl | oscseq.com | smadsteck.nl
_______________________________________________
OSC_dev mailing list
http://lists.create.ucsb.edu/mailman/listinfo/osc_dev

[Sciss]

the question for me is, we are moving to more capable systems, as andy points out. the trend or future as i see it is to move towards hardware abstraction. so i don't get it -- why would you want an unsigned 24 bit integer? and there not even a boolean type (or is that 'bit int' -- never heard that name).

if i design a system, the system spec is derived from future applications i want that system to perform. so it makes me wonder if there hasn't been a confusion here.

if someone gives me an example where unsigned 24 bit integers are useful, and are so useful that using a signed or unsigned 32 bit int instead is an unacceptable penality, i'm willing to think about it again. but i just don't see that case. where is a float 16 so useful that you cannot use a float 32?

why do you need a complex type if you can send real and imaginary as a tuple?

there is a reason that in any programming language more specialized types are composed of other types, and the number of primitives is usually really small. you don't need a vector a matrix primitive. a vector is just a compound object carrying an array of element primitives and, if you want, an int for the number of elements in the vector. and so forth. so all you need to minimize type tags is a tag for an array of elements of the same type so you can avoid sending the type tag for each element.

that is to say, if the receiver expects a message "/select" to come with an interval of two doubles, why bother about creating an interval primitive type when you can just use two doubles. if the receiver doesn't know the purpose of "/select", it can still happily forward that message to any interested party until there is one which knows what to do with it.

one should concentrate on the cases which couldn't yet be properly encoded. this for me are strings whose character encoding was completely unspecified. OSC 1.0 also lacked 64 bit types (although they were already optionally defined). ok, add 128 bit ones as well, if there is a need. but that's it. why make it unnecessarily complicated? adding complexity where it is not needed is a recipe for failure. the smaller the spec the better. that's why i'm happy that the types for RGB colours etc. were dropped -- because you can already unambiguously encode them with other types.

best, -sciss-

Post by Mattijs Kneppers
Hey sciss,
As long as a developer is free to skip the messages with data types
his program doesn't support, I can't see anything wrong with setting a
standard to support more data types, right?
A more relevant question as I see it is whether actively skipping data
in general (which needs parsing it first, at least to a certain
extent) is something we could try to avoid.
I don't remember whether it is mentioned in the available docs that
sending data to an ip and port should be limited to only the data
relevant for that device. Perhaps a guarantee for that could be looked
into, although I must admit I don't have ideas about how that could be
achieved.
Best,
Mattijs

Post by Sciss
the question is if any of those are particularly useful.
i'm more baffled by the fact there was no clear definition as how strings are encoded -- you propose UTF-8 now, which is great. that's a much more relevant decision IMO than adding half a dozen extra primitive types that have so little use in 98% of the applications, that requesting to support them in any implementation is a waste of resources. i think OSC 1.1 was good here with the four types, int32, long64, float32, double64.
best, -sciss-

Post by Adrian Freed

Post by s***@xs4all.nl
Andy, if I might just say - that looks great from a user point of view,
but having to implement all of those types would be a nightmare from a
developer point of view. Why the need for so many different integer
lengths, and so many different fixed point values ? I would stick to fewer
types as a developer, e.g.

I am afraid the ship has already sailed on this one.
http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0344h/Beidjgca.html
Application libraries (mainly in graphics) have things like half-floats.
I don't think these are particularly challenging for developers but we shall see as we prototype these things...
_______________________________________________
OSC_dev mailing list
http://lists.create.ucsb.edu/mailman/listinfo/osc_dev

_______________________________________________
OSC_dev mailing list
http://lists.create.ucsb.edu/mailman/listinfo/osc_dev

--
arttech.nl | oscseq.com | smadsteck.nl
_______________________________________________
OSC_dev mailing list
http://lists.create.ucsb.edu/mailman/listinfo/osc_dev

[Adrian Freed]

Post by Sciss
that is to say, if the receiver expects a message "/select" to come with an interval of two doubles, why bother about creating an interval primitive type when you can just use two doubles. if the receiver doesn't know the purpose of "/select", it can still happily forward that message to any interested party until there is one which knows what to do with it.

I tried to use this argument to resist the introduction of any type tags into OSC in the early days. I was shouted down by Matt Wright and James Macartney.

Post by Sciss
one should concentrate on the cases which couldn't yet be properly encoded. this for me are strings whose character encoding was completely unspecified. OSC 1.0 also lacked 64 bit types (although they were already optionally defined). ok, add 128 bit ones as well, if there is a need. but that's it. why make it unnecessarily complicated? adding complexity where it is not needed is a recipe for failure. the smaller the spec the better. that's why i'm happy that the types for RGB colours etc. were dropped -- because you can already unambiguously encode them with other types.

This is a difficult design problem that is reflected in the difficulty agreeing on where to draw the line between primitive and derived types. A good
discussion of this is welcome and Andy and I hoped the OSC 2.0 paper would prompt such a discussion.

The wikipedia entry is pretty sloppy on this but here is a relevant quote:

"Classic basic primitive types may include:
• Character (character, char);
• Integer (integer, int, short, long, byte) with a variety of precisions;
• Floating-point number (float, double, real, double precision);
• Fixed-point number (fixed) with a variety of precisions and a programmer-selected scale.
• Boolean, logical values true and false.
• Reference (also called a pointer or handle), a small value referring to another object's address in memory, possibly a much larger one."


References are problematic in OSC and get in the way of stateless programming.
Interest how this list above doesn't include strings or blobs which have been fairly "primitive" for a long time (e.g. C, SQL).

[Sciss]

hi adrian,

Post by Adrian Freed

Post by Sciss
that is to say, if the receiver expects a message "/select" to come with an interval of two doubles, why bother about creating an interval primitive type when you can just use two doubles. if the receiver doesn't know the purpose of "/select", it can still happily forward that message to any interested party until there is one which knows what to do with it.

I tried to use this argument to resist the introduction of any type tags into OSC in the early days. I was shouted down by Matt Wright and James Macartney.

i think this is a very different issue here. what you don't want is move the decoder logic into the application logic, and that is precisely what will happen when there are no type tags. you need to recognise that every typed language has representations for primitive values, and hence you can perfectly juggle around decoded messages in max or supercollider or any other language, and eventually when they arrive at the interested part, you can do a type check on the already decoded message. in pd or max this is `[route "/select", f, f]`, in scala this could be `case osc.Message( "/select", start: Double, stop: Double )`. so architecturally (for modularity, separation of concerns and type safety), type tags make big sense, and that doesn't mean there should be superfluous types.

another reason why type tags make sense is that you can have polymorphism (or call it message overloading).

a third reason is there can be more dynamically typed messages for which the receiver has no preexisting template. for example, in my SwingOSC application, i am matching the received arguments against method signatures of a language. so all i know is that a message ["/method", ref, ... ] invokes a method on a reference. which method is determined from the types, something any interpreter or compiler would do.

a fourth reason would have been that there are singleton types, but obviously you wanted to get rid of them (nil, true, false, ...)

Post by Adrian Freed

Post by Sciss
one should concentrate on the cases which couldn't yet be properly encoded. this for me are strings whose character encoding was completely unspecified. OSC 1.0 also lacked 64 bit types (although they were already optionally defined). ok, add 128 bit ones as well, if there is a need. but that's it. why make it unnecessarily complicated? adding complexity where it is not needed is a recipe for failure. the smaller the spec the better. that's why i'm happy that the types for RGB colours etc. were dropped -- because you can already unambiguously encode them with other types.

This is a difficult design problem that is reflected in the difficulty agreeing on where to draw the line between primitive and derived types. A good
discussion of this is welcome and Andy and I hoped the OSC 2.0 paper would prompt such a discussion.

sorry, i missed that one. i can see an "an excerpt from the draft document" for OSC 2.0 which is a bullet list posted by andy schmeder, but the actual draft wasn't there? if i have overseen it, can you repost the link, please!

Post by Adrian Freed
• Character (character, char);
• Integer (integer, int, short, long, byte) with a variety of precisions;
• Floating-point number (float, double, real, double precision);
• Fixed-point number (fixed) with a variety of precisions and a programmer-selected scale.
• Boolean, logical values true and false.
• Reference (also called a pointer or handle), a small value referring to another object's address in memory, possibly a much larger one."
References are problematic in OSC and get in the way of stateless programming.
Interest how this list above doesn't include strings or blobs which have been fairly "primitive" for a long time (e.g. C, SQL).

i agree. but also i think the computer world has moved forward from C. i just think it is a very biased hardware view to include 30 (!) primitives for all sorts of combinations of fixed-size ints and floats, but not even waste a thought whether other application areas could benefit lets say from a arbitrary-precision int or decimal (i'm sure the maths people, erlang, python, haskell, groovy, <your favourite language here> programmers would be happy...).

best, -sciss-

[Andy W. Schmeder]

Post by Sciss

Post by Adrian Freed
This is a difficult design problem that is reflected in the difficulty agreeing on where to draw the line between primitive and derived types. A good
discussion of this is welcome and Andy and I hoped the OSC 2.0 paper would prompt such a discussion.

sorry, i missed that one. i can see an "an excerpt from the draft document" for OSC 2.0 which is a bullet list posted by andy schmeder, but the actual draft wasn't there? if i have overseen it, can you repost the link, please!

I'm pretty sure he meant to say the OSC 1.1 paper (on the website)



---

Andy W. Schmeder
email: andy [at] cnmat.berkeley.edu
skype: andy.schmeder

Programmer/Analyst II
Research Group
Center for New Music and Audio Technologies
University of California at Berkeley
http://cnmat.berkeley.edu

[Andy W. Schmeder]

Post by Mattijs Kneppers
As long as a developer is free to skip the messages with data types
his program doesn't support, I can't see anything wrong with setting a
standard to support more data types, right?

Provided that freedom doesn't result in anarchy. :)


---

Andy W. Schmeder
email: andy [at] cnmat.berkeley.edu
skype: andy.schmeder

Programmer/Analyst II
Research Group
Center for New Music and Audio Technologies
University of California at Berkeley
http://cnmat.berkeley.edu

[Gaspard Bucher]

On Sun, Nov 6, 2011 at 11:20 PM, Andy W. Schmeder

Post by Andy W. Schmeder

Post by Mattijs Kneppers
As long as a developer is free to skip the messages with data types
his program doesn't support, I can't see anything wrong with setting a
standard to support more data types, right?

Provided that freedom doesn't result in anarchy. :)
---
Andy W. Schmeder
email: andy [at] cnmat.berkeley.edu
skype: andy.schmeder
Programmer/Analyst II
Research Group
Center for New Music and Audio Technologies
University of California at Berkeley
http://cnmat.berkeley.edu
_______________________________________________
OSC_dev mailing list
http://lists.create.ucsb.edu/mailman/listinfo/osc_dev

On the subject of how many data types are needed or (more importantly) are
not needed, please have a look at Lua. They managed to create a programming
language (a very efficient and fast one) with just a few primitives:

* real number
* string
* table
* true/false
* nil
* userdata (C object)

As for bandwidth optimization, I would consider this a non-issue for OSC:
if the device needs a huge throughput, I think it should use some serial
communication (not something with urls and pattern matching like OSC), at
least up to some more powerful device which would compress/pre-process the
data before forwarding it to OSC compliant devices. I do not think a single
protocol (OSC) can serve these two scenarios well:

A. Inter-application compatibility, easy to let multiple devices work
together, feature discovery, etc.
B. Fastest and most efficient way to transmit data from sensors to data
acquisition devices.

Point B would require lots of compression flavors with every bit sizes for
ints. Point A requires minimal complexity and data types so that we do not
require translation units between every device to let them talk together.


Gaspard

[Jeff Koftinoff]

While in general I agree with you that a minimal set of types can be workable, is it really a problem for OSC v2 to be defined with more data types? A device doesn't have to support them all…

Regards,
Jeff

Post by Andy W. Schmeder

Post by Mattijs Kneppers
As long as a developer is free to skip the messages with data types
his program doesn't support, I can't see anything wrong with setting a
standard to support more data types, right?

Provided that freedom doesn't result in anarchy. :)
---
Andy W. Schmeder
email: andy [at] cnmat.berkeley.edu
skype: andy.schmeder
Programmer/Analyst II
Research Group
Center for New Music and Audio Technologies
University of California at Berkeley
http://cnmat.berkeley.edu
_______________________________________________
OSC_dev mailing list
http://lists.create.ucsb.edu/mailman/listinfo/osc_dev
* real number
* string
* table
* true/false
* nil
* userdata (C object)
A. Inter-application compatibility, easy to let multiple devices work together, feature discovery, etc.
B. Fastest and most efficient way to transmit data from sensors to data acquisition devices.
Point B would require lots of compression flavors with every bit sizes for ints. Point A requires minimal complexity and data types so that we do not require translation units between every device to let them talk together.
Gaspard
_______________________________________________
OSC_dev mailing list
http://lists.create.ucsb.edu/mailman/listinfo/osc_dev

[Adrian Freed]

Post by Sciss
the question is if any of those are particularly useful.
i'm more baffled by the fact there was no clear definition as how strings are encoded -- you propose UTF-8 now, which is great. that's a much more relevant decision IMO than adding half a dozen extra primitive types that have so little use in 98% of the applications, that requesting to support them in any implementation is a waste of resources. i think OSC 1.1 was good here with the four types, int32, long64, float32, double64.

Yes, those types were chosen in the RISC days. CISC ended up trumping RISC eventually - to my surprise - not for architectural reasons, but for power consumption: a lot of the small
types (half floats etc.) have a big positive impact on power/performance numbers and they now pervade processor designs from portable devices to desktop machines.

As for your 98% number: we probably have different perspective on what the space of OSC applications is. Over 98% of my recent uses of OSC (e.g. piezo resistive multitouch) are all hitting performance walls unless I use smaller data types in the communications. They get upcast to other types at the destination but I need to express what is happening at the sensor array compactly.

[Andy W. Schmeder]

Post by Sciss
i'm more baffled by the fact there was no clear definition as how strings are encoded -- you propose UTF-8 now, which is great.

I am proposing UTF-8 for "s" type as well as allowing UTF-8 in the address strings (minus the few reserved characters).

Also there will be a way (through the meta-data / enumeration protocol) to indicate that a particular string encoded in UTF-8 also conforms with another standard, such as ISO8601 (a date-time string), ISO9899 (C literal numbers in ASCII), ISO9899:1999 (C99 literals).

Post by Sciss
that's a much more relevant decision IMO than adding half a dozen extra primitive types that have so little use in 98% of the applications, that requesting to support them in any implementation is a waste of resources. i think OSC 1.1 was good here with the four types, int32, long64, float32, double64.

Actually to be more clear int32 etc are not primitive types, they are machine encodings for primitive types.

ISO/IEC 11404:2007 provides a better reference for the underlying type system that I want OSC to capture, which of the numeric ones are:

boolean
integer
enumerated
scaled
rational
real
time
complex


---

Andy W. Schmeder
email: andy [at] cnmat.berkeley.edu
skype: andy.schmeder
mobile: [phone]

Programmer/Analyst II
Research Group
Center for New Music and Audio Technologies
University of California at Berkeley
http://cnmat.berkeley.edu

[Matt Wright]

I don't understand these recent comments about the supposed lack of specification of the encoding of an OSC-string.

OSC-string
A sequence of non-null ASCII characters followed by a null, followed by 0-3 additional null
characters to make the total number of bits a multiple of 32.

It's not UTF-8 (which I agree would today be a better choice). It's not Unicode at all. But the definition seems completely unambiguous to me. Am I missing something?

-Matt

i'm more baffled by the fact there was no clear definition as how strings are encoded -- you propose UTF-8 now, which is great
one should concentrate on the cases which couldn't yet be properly encoded. this for me are strings whose character encoding was completely unspecified.
I am proposing UTF-8 for "s" type as well as allowing UTF-8 in the address strings (minus the few reserved characters).
Also there will be a way (through the meta-data / enumeration protocol) to indicate that a particular string encoded in UTF-8 also conforms with another standard, such as ISO8601 (a date-time string), ISO9899 (C literal numbers in ASCII), ISO9899:1999 (C99 literals).
One example raised in this thread
was clarifying UTF8 as the standard string character format.

[Andy W. Schmeder]

Post by Matt Wright

Post by Matt Wright
OSC-string
A sequence of non-null ASCII characters followed by a null, followed by 0-3 additional null
characters to make the total number of bits a multiple of 32.

It's not UTF-8 (which I agree would today be a better choice). It's not Unicode at all. But the definition seems completely unambiguous to me. Am I missing something?

You are correct. And I concluded UTF-8 was the best choice of the various options specifically because its backwards compatible with ASCII. It also won't interfere with any of the OSC reserved characters because all UTF-8 sequences are in extended ASCII (high bit set).


---

Andy W. Schmeder
email: andy [at] cnmat.berkeley.edu
skype: andy.schmeder

Programmer/Analyst II
Research Group
Center for New Music and Audio Technologies
University of California at Berkeley
http://cnmat.berkeley.edu

[Sciss]

allowing strings only with byte characters in the ascii 7 bit range is certainly a non-definition of a string in a contemporary setting; it's probably useful to send the name of a midi instrument, but otherwise not. and clearly the implementations of OSC i have seen so far use all sorts of 8 bit encodings (from native (e.g. Mac Roman, Windows) to UTF-8 where a character can take more than a byte).

best, -sciss-

Post by Matt Wright
I don't understand these recent comments about the supposed lack of specification of the encoding of an OSC-string.

Post by Matt Wright
OSC-string
A sequence of non-null ASCII characters followed by a null, followed by 0-3 additional null
characters to make the total number of bits a multiple of 32.

It's not UTF-8 (which I agree would today be a better choice). It's not Unicode at all. But the definition seems completely unambiguous to me. Am I missing something?
-Matt

Post by Matt Wright
i'm more baffled by the fact there was no clear definition as how strings are encoded -- you propose UTF-8 now, which is great
one should concentrate on the cases which couldn't yet be properly encoded. this for me are strings whose character encoding was completely unspecified.
I am proposing UTF-8 for "s" type as well as allowing UTF-8 in the address strings (minus the few reserved characters).
Also there will be a way (through the meta-data / enumeration protocol) to indicate that a particular string encoded in UTF-8 also conforms with another standard, such as ISO8601 (a date-time string), ISO9899 (C literal numbers in ASCII), ISO9899:1999 (C99 literals).
One example raised in this thread
was clarifying UTF8 as the standard string character format.

_______________________________________________
OSC_dev mailing list
http://lists.create.ucsb.edu/mailman/listinfo/osc_dev

[Andy W. Schmeder]

Post by Sciss
allowing strings only with byte characters in the ascii 7 bit range is certainly a non-definition of a string in a contemporary setting;

I think you mean to say that its not an internationalized string encoding.

ASCII is a string encoding for english text and it is well-defined.


---

Andy W. Schmeder
email: andy [at] cnmat.berkeley.edu
skype: andy.schmeder

Programmer/Analyst II
Research Group
Center for New Music and Audio Technologies
University of California at Berkeley
http://cnmat.berkeley.edu

[Andy W. Schmeder]

Post by s***@xs4all.nl
Andy, if I might just say - that looks great from a user point of view,
but having to implement all of those types would be a nightmare from a
developer point of view. Why the need for so many different integer
lengths, and so many different fixed point values ? I would stick to fewer
types as a developer, e.g.

Good point, I will take a hard look at the debatable necessity for fixed point and unsigned ints, since at the least this requires a bunch of extra conversion routines. The main application for these is video, audio and embedded data acquisition systems.

The intention with that list was to cover every conceivable primitive type at the outset, but I have to agree its certainly preferable to cut things back as far as possible.


---

Andy W. Schmeder
email: andy [at] cnmat.berkeley.edu
skype: andy.schmeder

Programmer/Analyst II
Research Group
Center for New Music and Audio Technologies
University of California at Berkeley
http://cnmat.berkeley.edu

[Ross Bencina]

Post by Andy W. Schmeder
The plan for OSC 1.1 was never more than to patch
up some of the most glaring problems.
I think that to have OSC devices working well with
a broad range of applications and connectivity
methods requires a lot more effort and probably some
improvements to the core format, in particular better
type support and the functional "glue layers", i.e.,
discovery, enumeration and control. Possibly the
main issue is implementing and testing all that
which is considerable...

Andy,

I have some concerns about the above. I'm wondering if you could clarify
please.

[Andy W. Schmeder]

My impression is that there are a number of "cleanups" that can be made to clarify the OSC 1 spec that would benefit interoperability of the existing and future installed base. One example raised in this thread was clarifying UTF8 as the standard string character format. I imagine there are others. Presenting this and other previously released information in a formal specification document is desirable.

Noted.

"we don't care about supporting and nurturing the large installed base of OSC 1 applications, frameworks and devices, we're just going to get on with our future looking research agenda and make a non-backward-compatible OSC 2.0." This does not always end well.

Good point.

I think it is equally important that the OSC 1 spec is maintained and nurtured at least until such time as OSC 2.0 is out in the wild with multiple implementations.

I understand what you are getting at here; its convincing argument.

The obvious thing to do is harvest any of the items from my 2.0 todo-list that don't require format changes, then publish this as a 1.1 specification.

Also, later moving forward OSC 2.0 there will be better groundwork laid.

If CNMAT is unable to provide ongoing support for OSC 1 perhaps members of the community could be authorised to produce a maintenance release of the OSC 1 spec?

I think the preferences from CNMAT is that Adrian and I (et al) produce the 1.1 maintenance spec and then run that draft through a request for comments from the community prior to finalizing it. We are also interested in an updated "OSC-Kit", and we have looked at

By the way, its true that I work at CNMAT but my statements on this email list don't necessarily represent official CNMAT policy, which is decided by the directors.


---

Andy W. Schmeder
email: andy [at] cnmat.berkeley.edu
skype: andy.schmeder

Programmer/Analyst II
Research Group
Center for New Music and Audio Technologies
University of California at Berkeley
http://cnmat.berkeley.edu

[Adrian Freed]

Post by Andy W. Schmeder
I think the preferences from CNMAT is that Adrian and I (et al) produce the 1.1 maintenance spec and then run that draft through a request for comments from the community prior to finalizing it. We are also interested in an updated "OSC-Kit", and we have looked at

Yes, Andy let's dust off that last attempt at the spec.

Actually, Andy we have no formal RFC process. We read the developer group, we read e-mails sent to us, we talk at conferences, we listen at conferences. We share our code.

We also have no finalization process. This is a peculiar subtlety we learned more about recently that varies a lot between standardization efforts. Some actually finalize a standard and then
it becomes hard to change. Others always allow for refinements and clarifications (e.g. IEEE). Others never actually arrive at a standard (e.g. RFP and RFC's).
Some declare an effort a "non-standard".

Navigating this is going to be one of the first tasks for whomever attempts to move OSC 2.0 forward.

I would like these efforts to reflect actual good practice in the field rather than speculations as to what might be useful and to avoid the mess that for example RS232 became
with hardly a standard conforming connector/cable or implementation to be found anywhere.

Post by Andy W. Schmeder
By the way, its true that I work at CNMAT but my statements on this email list don't necessarily represent official CNMAT policy, which is decided by the directors.

Yes and I can confirm that OSC is regularly discussed by the directors.

[Adrian Freed]

From what I you say there will be no future updates or official
clarifications to the OSC 1 specification. Is that correct?

no, that's not correct.
We continue to update and clarify OSC 1. We have implemented various enhancements
to the original as outlined in our NIME paper and continue to test and refine them.
The next small but important step you will see is a clarification of how to describe OSC streams
in AVB.

[Jean-Baptiste Thiebaut]

On 7 Nov 2011, at 06:36, Adrian Freed wrote:

[Jeff Koftinoff]

Post by Adrian Freed

From what I you say there will be no future updates or official
clarifications to the OSC 1 specification. Is that correct?

no, that's not correct.
We continue to update and clarify OSC 1. We have implemented various enhancements
to the original as outlined in our NIME paper and continue to test and refine them.
The next small but important step you will see is a clarification of how to describe OSC streams
in AVB.

I was under the impression that OSC was removed from the AVB specs? There's no mention of OSC in the latest draft as far as I know.

While my original proposal to the 1722.1 groups was called AVBC and was completely based on OSC, the IEEE 1722.1 draft 16+, does not use OSC. One of the reasons I could not just refer to OSC from 1722.1 is that OSC is technically not a standard from IEEE, IETF, AES, ISO, ANSI, etc. Another reason why I did not use OSC eventually is because 1722.1 is now primarily a layer 2 protocol and is more in line with existing firewire protocols.

However in 1722a we are adding the time sensitive "AVB Control Streams" which would allow for the transport of OSC over AVB streams in a standard way - The "Control Stream Format", which is an EUI-64, would be UC Berkeley OUI-24 with the lower 40 bits chosen by UC Berkeley to mean "OSC V1", "OSC V1.1", "OSC V2" etc.

That said, I expect that the 'Control' aspect of 1722.1 would be able to be mapped to/from OSC fairly easily.

Regards,
Jeff

[Adrian Freed]

Thanks for the clarifications to the OSC developer group, Jeff.

Post by Jeff Koftinoff
However in 1722a we are adding the time sensitive "AVB Control Streams" which would allow for the transport of OSC over AVB streams in a standard way - The "Control Stream Format", which is an EUI-64, would be UC Berkeley OUI-24 with the lower 40 bits chosen by UC Berkeley to mean "OSC V1", "OSC V1.1", "OSC V2" etc.

This part is what OSC (and other show control protocols) has needed all along: end-to-end timely delivery and we will be forever grateful when this is broadly available.
We will focus on a freely available developer kit for OSC when the API's for 1722a are accessible on the desktop platforms - it looks like Linux and OS/X are well on their way.
Is anyone looking at this on Windows?

[Adrian Freed]

if I'm wrong, but observing from the outside you appear to be saying "we
don't care about supporting and nurturing the large installed base of
OSC 1 applications, frameworks and devices, we're just going to get on
with our future looking research agenda and make a
non-backward-compatible OSC 2.0." This does not always end well.

Please don't put words in our mouths.
I already explained how we put the equivalent of more than one full-time person's timeinto
OSC support - thanks to the generous support of our various CNMAT sponsors.

If CNMAT is unable to provide ongoing support for OSC 1 perhaps members
of the community could be authorised to produce a maintenance release of
the OSC 1 spec?

We do provide ongoing support for OSC 1 and we will publish a revised specification.
The community can help by testing implementations of it, e.g., the version John is about to release
for Max 6.

[Ross Bencina]

Post by Adrian Freed

if I'm wrong, but observing from the outside you appear to be saying "we
don't care about supporting and nurturing the large installed base of
OSC 1 applications, frameworks and devices, we're just going to get on
with our future looking research agenda and make a
non-backward-compatible OSC 2.0." This does not always end well.

Please don't put words in our mouths.

I'm sorry Adrian, I didn't mean any offence, but reading this thread
there has not been clear communication from CNMAT regarding this matter.
Kaspar asked a very clear question as to the whereabouts of the 1.1 spec
mooted in 2009. In response, Andy changed the subject and started
talking about 2.0.

Post by Adrian Freed

If CNMAT is unable to provide ongoing support for OSC 1 perhaps members
of the community could be authorised to produce a maintenance release of
the OSC 1 spec?

We do provide ongoing support for OSC 1 and we will publish a revised specification.

Great. Thanks for clarifying.