[lsb-discuss] LSB conf call notes for 2008-06-18

Submitted by Jeff Licquia on Wed, 06/18/2008 - 16:12.

Licquia, Robert Schweikert, Marvin Heffler, Alexey Khoroshilov, Ron Hale-Evans, Kay Tate, Ted Tso. Some glitches found in the 3.2 release; please report any others to the list or IRC. Working on new status table, at the top of ProjectPlan40 page on the wiki. Not done yet, but people can take a look at it and add data they think might be missing. gnu_get_libc_release. Do we need it? Robert: if it's LSB, then you shouldn't. Jeff: some apps may dlopen() extra interfaces. Kay: defensive purposes; guard against changes in glibc behavior. Robert: who uses it? Jeff: many different apps are in the Navigator. Mats: apps trying to distinguish glibc behaviors; LSB reduces the need for that. Dan: may be necessary as a migration aid, for vendors who have to support LSB and other situations. Jeff: on all distros. Ted: also, what if people implement the LSB using something besides glibc? Mats: we can only document the form of the data, not its content. Ted: can't really guarantee that the data returned is useful. Jeff: heavy use by proprietary apps. Mats: not surprising, given how easy it is to rebuild. Kay: also, those apps may be more finely tuned. Robert: thought they needed this to use an unstable interface, that had changed between glibc releases. This put them outside the LSB, and not always helpful, as the interface can continue to change. Ted: how harmful is the interface? What's the cost? Not very great. Jeff: research? Ted: votes for adding it anyway, to remove any excuses. Mats: maybe stall; is very easy to add later. Other libcs don't emulate this. Ted: would the added information change our decision or make it easier? __getdelim. Robert: avoid __ interfaces generally. Jeff: policy decision, do we include __ when there's an optimization to be had? Ted: do we know that this is just an optimization? Jeff: bug seems to indicate that. Really really really want. Robert, Kay: how do you know? Jeff: ISV demand, bug reports that things are slow that can be traced to this. Ted: makes sense, probably want to add a note to the database about __getdelim and then wait for someone to express a need. sendfile. Talked about this before. Jeff: summarize past discussions. Seems like a slam-dunk to include; any objections? Ted: man page is good, should use that. epoll. Seems to be consensus. Disagree? Mats: stable? Ted: yes. Jeff: No objection; should go in. remap_file_pages. Jeff: bug says we need more input. Robert: give people enough rope? Mats: going down the road of "more Linux-specific"; if we're comfortable with that, then should be OK. Ted: more of an efficiency gain; the same functionality can be had with mmap() as specified. Jeff: some efficiency gain. Ted: should be stable. Jeff: tend to feel we should include these kinds of things. getifaddrs. Robert: ugly structure brought in. Jeff: worried about IPv6 issues. More research? Mats: might be nice. Ted: the functionality is needed. Is there another way? Mats: getaddrinfo? Jeff: doesn't provide the same information. Ted: badly done, but legacy software uses this. Leaning towards including it. Mats: is there a correlation between these interfaces? Would it buy us anything? Are these the only interfaces that exclude apps? Ted: probably should include it, but do the research also. Mats: next three done already. (bugs 2153, 2154, 2155). Can skip next time.

[lsb-discuss] LSB conf call notes for 2008-06-18
Submitted by Joseph Kowalski on Wed, 06/18/2008 - 20:15.

Jeff Licquia wrote:
> __getdelim. Robert: avoid __ interfaces generally. Jeff: policy
> decision, do we include __ when there's an optimization to be had? Ted:
> do we know that this is just an optimization? Jeff: bug seems to
> indicate that. Really really really want. Robert, Kay: how do you
> know? Jeff: ISV demand, bug reports that things are slow that can be
> traced to this. Ted: makes sense, probably want to add a note to the
> database about __getdelim and then wait for someone to express a need.
>

If there is a need for something called "__getdelim", perhaps you should
create a public interface called "getdelim"?

If the LSB were to produce a specification for __getdelim, the LSB would
be in violation of POSIX.... Remember that stuff about "symbols beginning
with `_[_A-Z]' belonging to the implementation".

Even if there is an optimization to be had here, perhaps the implementation
believes that its not stable?

(Again, terse summary, but maybe this is what Ted is leading to.)

- jek3

[lsb-discuss] LSB conf call notes for 2008-06-18
Submitted by Jeff Licquia on Wed, 06/18/2008 - 20:15.

Joseph Kowalski wrote:
> If there is a need for something called "__getdelim", perhaps you should
> create a public interface called "getdelim"?

That's already been done.

> If the LSB were to produce a specification for __getdelim, the LSB would
> be in violation of POSIX.... Remember that stuff about "symbols beginning
> with `_[_A-Z]' belonging to the implementation".

We already do specify some _ and __ symbols, largely because the
compiler emits them on our behalf and they must exist for binary
compatibility. I think these are usually marked specially so they can't
be used directly in apps.

Our policy to date has been to avoid specifying these unless absolutely
necessary; the question was whether that policy should be amended to
cover strictly unnecessary, but handy, optimizations.

> Even if there is an optimization to be had here, perhaps the implementation
> believes that its not stable?

If they do, they're not showing it by emitting references to it in
binaries, which is what they're doing with this optimization.

[lsb-discuss] LSB conf call notes for 2008-06-18
Submitted by Denis Silakov on Wed, 06/18/2008 - 21:30.

On Wednesday 18 June 2008 11:19:57 pm Jeff Licquia wrote:
> Joseph Kowalski wrote:
> > If there is a need for something called "__getdelim", perhaps you should
> > create a public interface called "getdelim"?
>
> That's already been done.

Well, as Jeff has mentioned in bug 2128
(http://bugs.linuxbase.org/show_bug.cgi?id=2128), both getdelim and
__getdelim in glibc are weak aliases for _IO_getdelim.

> > If the LSB were to produce a specification for __getdelim, the LSB would
> > be in violation of POSIX.... Remember that stuff about "symbols
> > beginning with `_[_A-Z]' belonging to the implementation".

However, it's a normal way for glibc to introduce a 'normal' symbol as a weak
alias for the same symbol with the leading '__', which usually provides the
actual implementation (situation with getdelim is not exactly the same, but
getdelim is not a common example, indeed). Surely, glibc inside its own code
uses underscored symbols, not their aliases. On binary level, all symbols are
visible; but the weak aliases have 'weak' binding, while the symbols that
really implement the functionality, have the 'global' one.

The particular problem with getdelim comes from getline optimization; let me
quote the bug 2128 - here is the code from bits/stdio.h:

# ifdef __USE_GNU
/* Like `getdelim', but reads up to a newline. */
__STDIO_INLINE _IO_ssize_t
getline (char **__lineptr, size_t *__n, FILE *__stream)
{
return __getdelim (__lineptr, __n, '\n', __stream);
}
# endif /* GNU */

- we have getline function redeclared as inline and calling __getdelim. Thus,
if some application uses getline, it actually obtains __getdelim dependency.

--
Regards,
Denis.

[lsb-discuss] LSB conf call notes for 2008-06-18
Submitted by tytso on Thu, 06/19/2008 - 20:00.

On Thu, Jun 19, 2008 at 01:22:44AM +0400, Denis Silakov wrote:
> The particular problem with getdelim comes from getline
> optimization; let me quote the bug 2128 - here is the code from
> bits/stdio.h:
>
> # ifdef __USE_GNU
> /* Like `getdelim', but reads up to a newline. */
> __STDIO_INLINE _IO_ssize_t
> getline (char **__lineptr, size_t *__n, FILE *__stream)
> {
> return __getdelim (__lineptr, __n, '\n', __stream);
> }
> # endif /* GNU */
>
> - we have getline function redeclared as inline and calling
> __getdelim. Thus, if some application uses getline, it actually
> obtains __getdelim dependency.

One of the things that worries me a little about what we're doing if
we don't use the __ symbol is whether in the long-term, glibc will
always keep the getline() symbol as an exported symbol, and whether it
will always do what the optimized version found in the header files
say they will do.

There are some interfaces where this happens already (i.e., stat), and
I could easily imagine the glibc folks claiming that they only support
programs that use the glibc header files, and if the glibc header
files implement getline() in terms of __getdelim(), they are under no
obligation to provide the getline() symbol in glibc. Worse would be
if there is bitrot and inconsistencies between what the header files
do for a function like getline() and what the weak symbol for
getline() does, such that an application program which is built using
the standard glibc and glibc header files behaves one way, and that
same application built against the LSB header files behaves another.

That being said, trying to keep our header files in sync in terms of
the inline functions in glibc's header files, isn't something that I'm
excited about. So far at least, glibc is keeping an ABI interface for
getline(), if all applications built against glibc would never use
that symbol (since they would use __getdelim instead). Let's cross
our figers and hope that continues to be true.

- Ted

[lsb-discuss] LSB conf call notes for 2008-06-18
Submitted by Wichmann Mats D on Wed, 06/18/2008 - 23:15.

Denis Silakov wrote:
> On Wednesday 18 June 2008 11:19:57 pm Jeff Licquia wrote:
>> Joseph Kowalski wrote:
>>> If there is a need for something called "__getdelim", perhaps you
>>> should create a public interface called "getdelim"?
>>
>> That's already been done.
>
> Well, as Jeff has mentioned in bug 2128
> (http://bugs.linuxbase.org/show_bug.cgi?id=2128), both getdelim and
> __getdelim in glibc are weak aliases for _IO_getdelim.
>
>>> If the LSB were to produce a specification for __getdelim, the LSB
>>> would be in violation of POSIX.... Remember that stuff about
>>> "symbols beginning with `_[_A-Z]' belonging to the implementation".
>
> However, it's a normal way for glibc to introduce a 'normal' symbol
> as a weak alias for the same symbol with the leading '__', which
> usually provides the actual implementation (situation with getdelim
> is not exactly the same, but getdelim is not a common example,
> indeed). Surely, glibc inside its own code uses underscored symbols,
> not their aliases. On binary level, all symbols are visible; but the
> weak aliases have 'weak' binding, while the symbols that really
> implement the functionality, have the 'global' one.

So just a level set since there were a few questions on this.

LSB is primarily a binary - ABI - specification. It does not even
specify a development environment. However, to be realistic, we're
also interested in the API which is used to get to the ABI.

There are some symbols and other artifacts that appear in only
one or the other. Mostly the __ symbols are internal implementation
and don't need to appear as part of the ABI: if an implementation,
such as glibc, uses these internally, but that usage is not
directly exposed to applications, then that's fine; LSB describes
how the publicly-visible symbols behave, not how libraries internally
cause that behavior to happen. But sometimes there is a strange
overlap: certain tricks in the compiler or library may cause these
osensibly internal symbols to appear in the symbol table of binaries.
In this case we may add them to LSB, but mark them as appearing only
in the binary specification, not the source specification - this
means they're legal in binaries, but source code may not program
to them directly. There are also a few where odd tricks cause
symbols which appear in the API to not appear in the binaries -
an example of this might be stat, which will magically be revectored
to binary symbol _xstat. In this case we'll say that interface is
in the source specification only, and also document _xstat, as being
in the binary specification only.

[lsb-discuss] LSB conf call notes for 2008-06-18
Submitted by Wichmann Mats D on Wed, 06/18/2008 - 20:15.

Let me clarify my point here:

> gnu_get_libc_release. >
> Jeff: heavy use by proprietary apps. Mats: not surprising, given how
> easy it is to rebuild.

For an app packaged with a distribution, it will have been built
against the version of glibc shipped with that distro, and will
not need to check for drastically newer or older versions of glibc.
Thus, you wouldn't expect to see that kind of runtime check there.
Instead, capability checking would be done in the configure script.

Thus, my claim is it's expected to see these used primarily by
applications *not* tied to a specific distribution.

[lsb-discuss] LSB conf call notes for 2008-06-18

Active forum topics